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ABSTRACT 



This report summarizes results from the 1994 National 
Assessment of Educational Progress (NAEP) , specifically those results 
concerning geography and U. S. history. The 1994 NAEP asked 4th-, 8th-, and 
12th-grade students a series of questions designed to assess their knowledge 
level and skills applications in specific subjects. This report provides an 
in-depth look at the type of tasks that made up the assessments in geography 
and U. S. history and how the students performed on those tasks. It makes 
extensive use of examples of student work and of exercise- level statistics in 
examining performance in different skills areas and on particular assessment 
exercises. Specific attention is given to the ways that students use the 
tools and resources of history and geography. The questions reflected the 
content and cognitive dimensions deemed essential for an understanding of 
these subjects. One content dimension included four themes: change and 
continuity in U.S. democracy: ideas, institutions, practices, and 
controversies; the gathering and interaction of peoples, cultures, and ideas; 
economic and technological changes and their relation to society, ideas, and 
the environment; and the changing role of the United States in the world. A 
second content dimension covers eight' time periods: Three Worlds and Their 
Meeting in the Americas (beginnings to 1607) ; Colonization, Settlement, and 
Communities (1607 to 1763) ; The Revolution and the New Nation (1763 to 1815) ; 
Expansion and Reform (1801 to 1861) ; Crisis of the Union: Civil War and 
Reconstruction (1850 to 1877) ; Development of Modern America (1865 to 1920) ; 
Modern America and the World Wars (1914 to 1945) ; and Contemporary America 
(1945 to present) . At every grade, the overall geography performance of 
males was higher than that of females; however, in U.S. history overall 
performances for males and females was significantly different at 12th grade 
only where males slightly outperformed females. In both subject areas, the 
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performance of White students was higher than that of Black or Hispanic 
students. (MJP) 
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Chapter 1 



Introduction 



The National Assessment of Educational Progress (NAEP) is mandated by the United 
States Congress to survey the educational accomplishments of U.S. students. For more 
than a quarter of a century, NAEP has assessed the educational achievement of fourth-, 
eighth-, and twelfth-grade students in selected subject areas, and as such, it is the only 
nationally representative and continuing assessment of what America’s students know 
and can do. NAEP assessments are based on content frameworks and specifications 
developed through a national consensus process involving teachers, curriculum experts, 
parents, and members of the general public. The frameworks are designed to reflect a 
balance among the emphases suggested by current instructional efforts, curriculum 
reform, contemporary research, and desirable levels of achievement. 

In 1994, NAEP conducted assessments in geography and U.S. history. Both are 
fields in which students are required to have a strong knowledge base of facts and 
concepts. Mastery of these subjects also involves the ability to use a variety of tools and 
resources as well as competence in a range of interpretive skills, including recall, 
analysis, judgment, application, and evaluation. Among the more important resources 
used in the study of history are primary source documents, such as the U.S. Constitution, 
the Declaration of Independence, personal correspondence, pictures, photographs, 
political cartoons, literature, and other artifacts. Historians interpret primary source 
documents by placing them in comparative, thematic, and chronological contexts to 
further their knowledge and understanding of historical events. Among the major tools of 
the field of geography are maps and atlases. Geographers use a variety of maps to guide 
their work and to record information. Other tools and resources in history and geography 
include charts, graphs, tables, and timelines. 

While achievement in history and geography involves both knowledge and skills, 
many past assessments in these areas have tended to focus on the former at the expense of 
the latter. Conversely, some other assessments have downplayed knowledge in order to 
emphasize the mastery of skills. The frameworks that guided the 1994 NAEP 
assessments, by contrast, attempted to achieve a more balanced view of the relationship 
between knowledge and skills. Specifically, both the geography and U.S. history 
frameworks portrayed their disciplines as fields in which factual knowledge, use of 
specialized tools and resources, and interpretive skills are all inseparable components 
of achievement. 
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Basic results from the 1994 NAEP assessments in geography and U.S. history 
have been presented in a series of reports released earlier. 1 These reports were intended 
for policymakers, educators, and the general public. They focused primarily on overall 
scale-score and achievement-level results for major populations in the United States, and 
on general factors related to achievement in history and geography. This current study has 
a more specific target audience (history and social studies educators) and a different 
purpose: a more in-depth look at the types of tasks that made up the 1994 NAEP 
assessments, and at how students performed on those tasks. Specifically, this report 
examines the ways in which students use the tools and resources of history and 
geography. Rather than looking at aggregated results, we examine performance in 
different skills areas and on particular assessment exercises. This report therefore makes 
extensive use of examples of student work and of exercise-level statistics. 

As was mentioned above, this report will examine the success students had 
working with a range of resource materials similar to those used by professional 
geographers and historians. History and geography educators view the ability to interpret 
a broad range of authentic materials as an essential element of learning in these fields. 2 
Many questions on both surveys were not based on the types of stimulus discussed in this 
report; for example, a number of tasks were designed to measure student knowledge and 
not the ability to interact with textual, quantitative, or graphic materials. Because of the 
particular focus of this report, no attempt is made to discuss the full range of knowledge 
and skills assessed as part of the 1994 NAEP geography and U.S. history assessments. 

1994 NAEP U.S. History Framework 

The framework for the 1994 NAEP U.S. history assessment 3 represented an ambitious 
vision both of what students should know and be able to do, and of the ways in which 
those competencies should be assessed. It presented the study of history as an exciting 
endeavor and emphasized the importance of knowing and understanding history in all its 
complexity — stressing the relationship between people, events, and ideas in 
understanding the past. Furthermore, the framework called for an assessment that 



1 Persky, H. R., Reese, C. M., O’Sullivan, C. Y., Lazer, S., Moore, J., & Shakrani, S. (1996). NAEP 1994 
geography report card. National Center for Education Statistics. Washington, DC: U.S. Government 
Printing Office. 

Beatty, A. S„ Reese, C. M„ Persky, H. R., & Carr, P. (1996). NAEP 1994 U.S. history report card. 
National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. 

Williams, P. L., Lazer, S„ Reese, C. M„ & Carr, P. (1995). NAEP 1994 U.S. history: A first look. National 
Center for Education Statistics. Washington, DC : U.S. Government Printing Office. 

Williams, P. L., Lazer, S„ Reese, C. M„ & Shakrani, S. (1995). NAEP 1994 geography: A first look. 
National Center for Education Statistics. Washington, DC : U.S. Government Printing Office. 

2 Educational Testing Service. (1987) U.S. history objectives; 1988 assessment. Princeton, NJ: Author. 
National Assessment Governing Board. (1992). U.S. history framework for the 1994 National Assessment 
of Educational Progress. Washington, DC: Author. 
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reflected the richness of history and historical sources through the use of a variety of 
grade-appropriate stimulus materials. 

The 1994 framework was organized around two content dimensions and one 
cognitive dimension. One content dimension focused on four themes which represent the 
major areas of endeavor that have characterized U.S. history. These four themes were: 

1. Change and Continuity in American Democracy: Ideas, Institutions, Practices, 
and Controversies 

2. The Gathering and Interaction of Peoples, Cultures, and Ideas 

3. Economic and Technolgical Changes and Their Relation to Society, Ideas, and 
the Environment 

4. The Changing Role of America in the World 

Because history is concerned with the experiences of people over time, it is 
critical to establish a basic chronological structure for tracing, reconstructing, and 
connecting the stories of those experiences. Thus, the second content dimension provided 
a chronological structure for the many issues included in the four themes. Eight periods 
were identified to focus attention on several major eras of U.S. history. They overlapped 
at some points in order to permit coherent coverage of major trends and events. The 
periods were: 

1. Three Worlds and Their Meeting in the Americas (Beginnings to 1607) 

2. Colonization, Settlement, and Communities (1607 to 1763) 

3. The Revolution and the New Nation (1763 to 1815) 

4. Expansion and Reform (1801 to 1861) 

5. Crisis of the Union: Civil War and Reconstruction (1850 to 1877) 

6. The Development of Modem America (1865 to 1920) 

7. Modem America and the World Wars (1914 to 1945) 

8. Contemporary America (1945 to present) 
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As Figure 1.1 illustrates, the themes and periods of U.S. history functioned as a 
matrix. The framework made clear that not all themes were equally important in each 
period. It also included special recommendations for adapting the assessment for fourth- 
grade students, who might not have received any formal instruction in U.S. history. 
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Note: Not all of the themes were addressed equally in every period. 



In addition to themes and periods, the U.S. history framework explicitly 
considered the ways of thinking and kinds of knowledge that historical study requires. 
These were divided into the following two general cognitive domains: 

1- Historical Knowledge and Perspective. This domain includes knowing and 
understanding people, events, concepts, themes, movements, contexts, and 
historical sources; sequencing events; recognizing multiple perspectives and 



4 Ibid. 
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seeing an era or movement through the eyes of different groups; and 
developing a general conceptualization of U.S. history. 

2. Historical Analysis and Interpretation. This domain includes explaining issues; 
identifying historical patterns; establishing cause-and-effect relationships; 
finding value statements; establishing significance; applying historical 
knowledge; weighing evidence to draw sound conclusions; making defensible 
generalizations; and rendering insightful accounts of the past. 

1994 NAEP Geography Framework 

The 1994 NAEP geography framework 5 reflected the heightened need for geographic 
knowledge and skills that has arisen as the world has become increasingly interconnected 
through technological advancement and shared concerns about economic, political, 
social, and environmental issues. The 1994 geography framework required students to 
reach far beyond place-name geography. It called for an assessment in which students 
would demonstrate an ability to work with the tools of geography, which include maps, 
aerial photographs, atlases, and graphs. The intent was to give students access to 
information conveyed through these tools, and ask them to use this information to 
understand and explain complex relationships and systems, such as ecosystems, 
communications networks, and urban infrastructures. In addition, students were expected 
to construct geographic representations, such as maps and diagrams, from narrative 
descriptions. 

Like the 1994 history assessment, the NAEP 1994 geography framework was 
organized by a matrix of two interrelated dimensions: content and the cognitive demands 
of the discipline. Content was divided into the following three areas corresponding to the 
major branches of geographic study: 

1. Space and Place. This area includes knowledge of geography as it relates to 
particular places on Earth, to spatial patterns on Earth’s surface, and to physical 
and human processes that shape such spatial patterns. 

2. Environment and Society. This area includes knowledge of geography as it 
relates to the interactions between environment and society. 

3.. Spatial Dynamics and Connections. This area includes knowledge of 
geography as it relates to spatial connections among people, places, and 
regions. 



5 National Assessment Governing Board. (1992). Geography framework for the 1994 National Assessment 
of Educational Progress. Washington, DC: Author. 
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The cognitive dimension of the framework specified the kinds of thinking 
expected of students as they deal with specific geography content. The dimension was 
organized into the following categories: 

1. Knowing. Tasks in this area are generally meant to measure students’ ability to 
observe different elements of the landscape and to answer questions by 
recalling information. 

2. Understanding. In this area, students are asked to attribute meaning to what has 
been observed and to explain events. 

3. Applying. This area of thinking calls on students to use many tools and skills of 
geography as they attempt to develop a comprehensive understanding of a 
problem en route to proposing viable solutions. 

Figure 1.2 illustrates the matrix formed by the content and cognitive dimensions 
of the assessment by presenting sample tasks and questions. The assessment addressed 
each cognitive process in each content area. 



Figure 1.2 


1 994 NAEP Geography Assessment 
Framework Dimensions 6 
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Applying* 


Support the conclusion How can both economic and Compare current settlement and 

that tropical rain forests environmental interests be employment patterns of Cuban 

promote wide speries reconciled in an area of and Mexican immigrants in the 

variation. strip mining? United States. 



Note: Example questions are illustrative only, and are not meant to represent the full array of assessment content. 
‘This dimension involved a range of higher-order thinking skills. 
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1994 NAEP Geography and U.S. History Assessments 

Guided by these new and more forward-looking frameworks, the 1994 NAEP 
assessments in geography and U.S. history shared a number of innovative and important 
characteristics. These included the following: 

• Both assessments used a wide range of authentic materials as stimuli for 
assessment questions. These included an atlas, maps, charts, graphs, tables, 
text-based primary source documents and literary works, and various art forms, 
including photographs, paintings, cartoons, and posters. Overall, 76 percent of 
the questions in the geography assessment and 56 percent in the history 
assessment involved working with such stimuli. 

• Both assessments assessed a range of skills related to these stimuli. In addition 
to straightforward interpretative exercises, students were frequently asked to 
synthesize information from multiple stimuli or to use outside knowledge in 
order to interpret a given stimulus. They might, for example, be asked both to 
describe the data in a table and to draw on outside knowledge to give factually 
accurate explanations for the patterns revealed. In this way, stimulus 
interpretation was not artificially separated from content knowledge. 

• Both assessments included performance tasks. In a number of exercises 
students were asked to create maps or graphs based on narratives or tables of 
quantitative data. 

• Because of the use of performance assessment and the requirement in the 
frameworks that the geography and history surveys measure broad ranges of 
content and skills, the overall assessments were too long for any one student to 
complete. For example, the grade 8 United States history assessment would 
have taken over four hours for an individual student. For these reasons, NAEP 
used a design in which participating students took only subsets of the aggregate 
item pool (specifically, in both geography and U.S. history individual students 
were tested for 50 minutes). 7 However, it is important to remember that a key 
feature of this design is that the samples of students who were presented each 
exercise were representative of the school population at a given grade. This 
design does not allow for the accurate computation of individual assessment 
scores; it does allow for the estimation of group performance on the assessment 
as a whole, on sets of questions, and on individual assessment exercises. 



7 Most students answered between 30 and 35 questions, which represents a subset of the total asked. The 
total number of questions in the assessment were 90, 125, and 123 at grades 4, 8, and 12 in geography ; and 
94, 148, and 156 at the three grades in U.S. history. 
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The 1994 NAEP assessments included multiple-choice questions, short 
constructed-response questions, and extended constructed-response questions. Table 1.1 
shows the distribution of questions by grade and format for both the 1994 NAEP 
geography and U.S. history assessments. 



Table l.l 


Number off Questions by Grade Level and 
Format in the 1 994 NAEP Geography 


THE h 
REPORT 
CARD 


lATlON’S 

rasp 

=k'i 




and U.S. History Assessments 









Geography 


U.S. History 


Grade 4 (total) 


90 


95 


Multiple-Choice 


59 


63 


Short Constructed- Response 


23 


26 


Extended Constructed-Response 


8 


6 


Grade 8 (total) 


125 


151 


Multiple-Choice 


84 


102 


Short Constructed- Response 


32 


37 


Extended Constructed-Response 


9 


12 


Grade 1 2 (total) 


123 


155 


Multiple-Choice 


85 


103 


Short Constructed- Response 


25 


33 


Extended Constructed-Response 


13 


19 


SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 994 Geography and U.S. History Assessments. 



For constructed-response questions, students provided written responses or 
performed tasks, such as constructing a graph. Each constructed-response question was 
scored according to a scoring guide that gave varying degrees of credit for correct or 
partially correct answers. Short constructed-response questions were scored according to 
three-level scoring guides in which a “Complete” score represented a complete and 
appropriate answer, a “Partial” score indicated that the response had some, but not all, of 
the components of an appropriate response, and an “Inappropriate” score represented an 
answer that had none of the components of an appropriate response. 

Extended constructed-response questions were lengthier and more complex 
exercises that allowed for a finer level of discrimination in scoring the responses. 
Responses were scored according to four-part scoring guides in which a “Complete” 
score was assigned to a response that was complete and appropriate; an “Essential” 
response was less complete but included the most important components of an 
appropriate response; a “Partial” response included some appropriate components, but 
fewer or less central ones than those required for an “Essential” score; and an 
“Inappropriate” response included only inappropriate material. 
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As with all NAEP assessments, the schools and students participating in the 1994 
geography and history assessments were selected through scientifically designed, 
stratified random sampling procedures. Approximately 19,000 fourth, eighth, and twelfth 
graders in 1,500 public and nonpublic schools across the country participated in the 1994 
geography assessment, and approximately 22,000 fourth, eighth, and twelfth graders in 
1,500 public and nonpublic schools participated in the 1994 U.S. history assessment. 
Detailed reports on the assessment procedures and results of these assessments are 
presented in two separate publications from the National Center for Education Statistics: 
NAEP 1994 U.S. History Report Card and NAEP 1994 Geography Report Card. 

Orientation of This Report 

To examine students’ ability to understand and use a variety of tools and resources, 
assessment questions were categorized according to the type of tool or resource, if any, 
that served as the stimulus — that is, the textual or graphic material provided to students 
which was to be considered in the formulation of their responses. Based on the nature of 
the stimulus, questions were placed into one of six categories: 

atlases 

maps 

primary source documents 
charts, graphs, and tables 
photographs 

art, which included paintings and cartoons. 

For this report, questions that did not use a stimulus, used more than one type of 
stimulus, or used a stimulus that could not easily be classified into one of the six 
categories established, were excluded from analytical consideration. 

Table 1.2 shows that — except for atlas questions (which were restricted to 
geography) and art questions (which were restricted to U.S. history) — the 1994 NAEP 
geography and U.S. history assessments both included questions in each of the different 
stimulus categories. It is also evident from the table, however, that some stimuli are more 
central to the study of one discipline than to another; for example, there were 
substantially more map questions in the geography assessment than in the U.S. history 
assessment, and more primary source documents in the U.S. history assessment than in 
the geography assessment. 
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One set of geography questions at each grade level was based on a Nystrom 
Classroom Atlas 8 The questions in the atlas block were all categorized as atlas questions, 
whether they employed maps, charts, or graphs as stimuli. The primary source document 
category was restricted to only those questions that utilized text-based primary sources. 
Although photographs and paintings also are kinds of primary sources, they were 
categorized separately. 

In addition to an introduction and a concluding chapter, this report includes five 
chapters corresponding to specific categories of stimuli (photography and art are 
discussed together in one chapter). Each chapter begins with a general description of the 
tool or resource. Teacher and student responses to NAEP questionnaires about classroom 
practices related to the relevant tools or resources are also provided in many of the 
chapters. 

For the two categories with the largest numbers of questions (maps and primary 
source documents), the descriptive introduction to the chapter is followed by a discussion 
of student performance on the group of stimulus questions as a whole and in comparison 
to all other questions in the geography or U.S. history assessment . 9 The remainder of each 
chapter is devoted to a detailed discussion of representative exercises. These exercises 
exemplify the skills that students were required to demonstrate in order to answer the 
questions included in the specific tool or resource category . 10 Information on how 




* World atlas: A resource for students. (1992 ed). (1990). Chicago, IL.: NYSTROM, Div. of Herff Jones. 
This comparison is limited to these two stimulus groups because the number of items included in the other 
four groups was too small to allow valid comparisons. 

10 Most of the questions in the 1994 NAEP assessments were not released for public review. Therefore, 
exercises shown in this report were chosen from only the poison of the 1994 NAEP surveys chosen for 
public release. <£» U 
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students performed on these individual questions is provided along with samples of actual 
student responses to constructed-response questions. Although there were questions from 
nearly all of the stimulus groups in each of the assessments, all but one of the chapters 
(charts, graphs, and tables) focuses on questions from the assessment in which they 
predominated. 

Although the report is structured around particular tools and resources, the reader 
should bear in mind that many factors besides the stimulus influence question difficulty. 
Both the content knowledge and the cognitive skills required to respond correctly are 
inextricably intertwined in the assessment task, as they are in the practice of the 
disciplines of geography and history. For example, the difficulty of most of the map 
questions may result from the demand that students use their geographical knowledge and 
a number of different map reading skills to correctly respond. Similarly in history, the 
difficulty of any particular item did not necessarily depend on the nature of the stimulus, 
but rather on the content and conceptual information students needed to have to 
understand, interpret, and respond fully to the question. 

Interpreting NAEP Results 

Student responses were analyzed to determine the percentage of students responding 
correctly to each multiple-choice question and the percentage of students responding in 
each of the score categories for constructed-response questions. Weighting procedures 
were then applied to arrive at overall population percentages and percentages for 
subgroups of students. The percentages are estimates because they are based on samples 
rather than on the entire population. As such, the results are subject to a measure of 
uncertainty that is reflected in the standard errors of the estimates. Standard errors provide 
a measure of how much survey results would be expected to vary if a different but equally 
valid sample of students were chosen. These standard errors are presented in parentheses 
along with the estimated percentage-correct scores in tables throughout this report." In 
the following chapters, all comparisons among question types or between subgroups of 
students are based on statistical tests that consider both the magnitude of the differences 
between the average percentages and their standard errors. Throughout this report, 
differences are discussed only when they are significant from a statistical perspective. 

This means that observed differences are unlikely to be due to chance factors associated 
with sampling variability. All differences are significant at the .05 level with appropriate 
adjustments made for multiple comparisons. The term “significant,” therefore, is not 
necessarily intended to imply judgment about the absolute magnitude or educational 



" The standard errors in this report should be interpreted in the following fashion: There is a 95 percent 
probability that a statistic for a population of interest is within two standard errors of the mean reported. 
For example, if we report that 50 percent of female students answered a question correctly and that the 
standard error is 0.5, then there is a 95 percent chance that the appropriate statistic falls between 49 and 
51 percent. 
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relevance of the differences. The term is intended to identify statistically dependable 
population differences as an aid in focusing subsequent dialogue among policy makers, 
educators, and the public. 

Overall Assessment Results 

Average percent-correct performance for the assessments as a whole is shown in Table 
1.3. Average percent correct, as used here, represents a different summary metric than the 
scale scores used in NAEP report-cards, and is designed to give readers a concrete sense 
of student performance on the specific exercises making up the assessments. Average 
percent correct is determined by obtaining the mean item score for each assessment 
question and averaging these over the full set of exercises. For multiple choice and 
dichotomously-scored constructed-response questions that is constructed-response 
questions scored on a two-part scale, the statistic represents the percentage of students 
who answered the question correctly. For polytomously-scored exercises that is, short and 
extended answered questions that are scored on either a three-point or four-point scale, 
the statistic represents the average score expressed as a percentage of the maximum 
possible score. Because the NAEP design ensures that each item is administered to a 
representative subset of the full sample, the averages presented in Table 1.3 provide a 
consistent estimate of the average item score that would be obtained if students were 
administered the entire assessment. 

At every grade, the overall geography performance of males was higher than that 
of females; however, in U.S. history overall performance for males and females was 
significantly different at twelfth-grade only, where males slightly outperformed females. 
In both subject areas, the performance of White students was higher than that of Black 
students and Hispanic students. 12 



22 

12 

There were insufficient sample sizes for the American Indian, Asian, and Pacific Islander racial/ethnic 
subgroups to produce reliable results. Consequently, racial subgroup information is only provided for 
White, Black, and Hispanic subgroups. 
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Table 1.3 


Average Item Score for 1994 NAIF 


THE N 
REPORT 
CARD 


lATION’S 


Geography and U.S. History Assessments 









Geography 


U.S. History 


Grade 4 


46 (0.5) 


41 (0.4) 


Male 


48 (0.6) 


40 (0.5) 


Female 


45 (0.6) 


41 (0.4) 


White 


51 (0.7) 


44 (0.5) 


Black 


31 (0.8) 


31 (0.5) 


Hispanic 


37(1.0) 


32 (0.9) 


Grade 8 


49 (0.3) 


44 (0.3) 


Male 


50 (0.4) 


44 (0.4) 


Female 


48 (0.4) 


44 (0.3) 


White 


54 (0.4) 


47 (0.4) 


Black 


34 (0.8) 


34 (0.5) 


Hispanic 


39 (0.9) 


36 (0.5) 


Grade 12 


50 (0.4) 


41 (0.4) 


Male 


52 (0.6) 


41 (0.4) 


Female 


48 (0.4) 


40 (0.4) 


White 


54 (0.5) 


43 (0.4) 


Black 


36 (0.7) 


30 (0.7) 


Hispanic 


41 (0.9) 


32 (0.6) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography and U.S. History Assessments. 



In the chapters that follow, statistically significant differences in performance by 
gender and racial/ethnic subgroups are noted only when these differences vary from those 
observed for the geography and U.S. history assessments as a whole. 
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Chapter 2 



Working with 

Primary Source Documents 

Recent pedagogical reforms have emphasized the importance of using primary source 
documents to educate students about history. 1 Such documents help students acquire 
factual knowledge about the past, understand the past from differing points of view, and 
think about how past events influence the present. Appropriately chosen primary source 
documents are valuable in presenting historical and geographical issues, events, and 
controversies more authentically and, in some cases, more clearly than are textbooks or 
other secondary treatments. For example, reading a letter or a journal entry of someone 
with first-hand experience of a time or event lends tremendous insight into the interaction 
between human experience and larger historical events. The use of primary source 
documents helps students to become critical thinkers who can analyze and interpret 
information and draw conclusions for themselves. Furthermore, since primary sources are 
the basic resources historians use to construct their view of the past, it is important for 

students to learn to analyze these sources as a way of formulating their own 

2 

interpretations of history. 

Classroom Instruction 

In the 1994 NAEP U.S. history assessment, students and their teachers were asked about 
various practices in history or social studies instruction. Both students and teachers 
reported much higher use of textbooks than of primary source documents or other extra 
written materials. 




1 Kobrin, D. (1996). Beyond the textbook: Teaching history using documents and primary sources. 
Portsmouth, NH: Heinemann. 

Wilson, D. W. (1993). Teaching for the future by reaching into the past. Teaching History: A Journal of 
Methods, 75(1), 3-13. 

2 Porter, P. H. (1994). The student as historian: Teacher’s resources. Social Studies and the Young Learner, 
7 (2), 23-26. 
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As shown in Table 2.1, the majority of fourth-grade students had teachers who 
reported that their students read textbooks at least once or twice a week. 3 However, when 
teachers were asked how frequently they used primary source documents, the modal 
response was never or hardly ever,” and when asked about the frequency of reading 
extra written materials beyond those included in textbooks, the modal response was “once 
or twice a month.” 

The situation at grade 8 was similar, but differed in important ways. The vast 
majority of eighth graders (87 percent) had teachers who reported assigning textbook 
reading at least once or twice each week. Eighth graders were substantially less likely 
than fourth graders to have teachers who reported never or hardly ever using primary 
source documents in history instruction; they were also more likely to report using such 
materials on a weekly basis. Most grade 8 students have teachers who report using 
primary documents once or twice each month. The use of extra written materials beyond 
those included in textbooks did not significantly vary between the two grades. 



Table 2.1 


Teachers' Reports on the Frequency of 
Use of Materials in Fourth-Grade and 
Eighth’Grade History Instruction 
(Percentage of Students) 


THE t 
REPORT 
CARD 


iATION’S 

mp 






Grade 4 


Grade 8 




Use of Primary Source Documents 

Almost Every Day 
Once or Twice a Week 
Once or Twice a Month 
Never or Hardly Ever 


1 (0.4) 
8(1.2) 
29(1.9) 
62 (2.0) 


2 (0.7) 
20 (2.2) 
55 (2.3) 
23(1.9) 




Use of Textbooks 

Almost Every Day 
Once or Twice a Week 
Once or Twice a Month 
Never or Hardly Ever 


43 (2.6) 

44 (2.6) 
8(1.4) 
5 (0.9) 


45 (3.6) 
42 (3.2) 
9(1.4) 
5(1.8) 




Use of Extra Written Materials 

Almost Every Day 
Once or Twice a Week 
Once or Twice a Month 
Never or Hardly Ever 


6(1.2) 
33 (2.2) 
46 (2.5) 
15(1.7) 


3 (0.6) 
33 (2.8) 
48 (2.4) 
17(2.2) 




Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 U.S. History Assessment. 



2 

Teachers of twelfth-grade students were not surveyed because twelfth-grade students are not necessarily 
enrolled in history classes. 
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As shown in Table 2.2, fourth-, eighth-, and twelfth-grade students reported 
similar patterns of use of these three types of instructional materials. Fourth- and eighth- 
grade students provided a somewhat different picture than teachers about the frequency 
with which primary source documents textbooks and extra written materials are used. 
However, there was general agreement between teachers and students about the frequency 
of their use of textbooks compared to their use of primary source documents and extra 
written materials. Both groups indicated that textbooks are used with greater frequency 
than the other two sets of materials. 



Table 2.2 


Percentage off Students Reporting on 
Frequency off Use off Materials in History 
Instruction by Grade Level 


THE NATION’S 
REPORT icaon 
CARD 

dS); 


Grade 4 Grade 8 Grade 1 2 



Use of Primary Source Documents 



Almost Every Day 


10(0.5) 


6 (0.3) 


4(0.2) 


Once or Twice a Week 


17(0.6) 


13(0.5) 


14(0.5) 


Once or Twice a Month 


19(0.7) 


22 (0.6) 


24 (0.5) 


A Few Times a Year 


17(0.8) 


24 (0.7) 


26 (0.5) 


Never 


38 (0.9) 


35 (0.8) 


32 (0.7) 



Use of Textbooks 



Almost Every Day 
Once or Twice a Week 
Once or Twice a Month 
A Few Times a Year 
Never 


46(1.5) 
25 (0.8) 
8 (0.5) 
8 (0.6) 
13(0.8) 


51 (1.3) 
33(1.1) 
8 (0.6) 
5 (0.5) 
3 (0.3) 


40 (0.9) 
40 (0.9) 
9 (0.5) 
6 (0.4) 
6 (0.5) 


Use of Extra Written Materials 

Almost Every Day 


12(0.5) 


1 1 (0.5) 


9 (0.4) 


Once or Twice a Week 


29 (0.9) 


29 (0.8) 


30 (0.8) 


Once or Twice a Month 


20 (0.6) 


25 (0.6) 


24 (0.5) 


A Few Times a Year 


15(0.6) 


1 9 (0.6) 


18(0.6) 


Never 


25 (0.8) 


17(0.6) 


18(0.7) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 U.S. History Assessment. 
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Primary Source Questions in the 
1994 NAEP U.S. History Assessment 

The 1994 NAEP U.S. history assessment contained a number of exercises that used 
primary source materials as stimuli. These materials included core political historical 
documents, such as the U.S. Constitution and the Declaration of Independence; orations 
such as Lincoln’s Gettysburg Address and Roosevelt’s inaugural speech; literary 
documents such as journal entries, poems, and songs; individual communications such as 
Nixon s letter of resignation and Chief Seattle’s letter to President Pierce regarding the 
selling of land; and other historical print materials such as newspaper reports, passenger 
manifests, and product labels. The text-based primary source questions from the 1994 
U.S. history assessment form the focus for this chapter; graphics-based primary sources 
such as photographs and paintings are included in Chapter 5. 

In most of the exercises using a primary source stimulus, students were 
presented with excerpts from primary source documents and were required to draw on 
their own knowledge of history to accomplish one or more of the following tasks: 

• Identify the primary source document, the author of the document, or the date 
or time period in which it was authored. 

• Read, understand, and explain the content or purpose of the primary source 
document, placing it in appropriate historical context. 

• Compare and contrast viewpoints expressed or represented in the primary 
document or documents. 

For example, when given an excerpt from a central historical document or a 
speech, students could be asked to identify the document or the speaker; or they could be 
asked to interpret the excerpt in terms of issues or controversies occurring at the time; or 
they might be asked to explain the position taken by the writer or speaker, making 
connections between the ideas expressed and the prevailing historical context. For many 
questions, students were asked to use a combination of these skills. In the 1994 NAEP 
U.S. history assessment, the total number of questions that required students to use 

primary source documents was 16 at fourth grade, 30 at eighth grade, and 47 at twelfth 
grade. 
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Student Performance on the 

1994 U.S. History Primary Source Questions 

As shown in Table 2.3, the assessment exercises in the fourth-grade U.S. history 
assessment that used primary source documents as stimuli were, as a group, less difficult 
than the other questions in the assessment. The difficulty of the primary source questions 
increased at the higher grades both in absolute terms and relative to other questions in the 
assessment. This may be due, in part, to the fact that eighth- and twelfth-grade students 
were expected to use the primary source materials in progressively more complex ways. 



Table 2.3 


Average Item Score for 


THE h 
REPORT 
CARD 


lATlON’S 

Kmp 


1994 NAEP U.S. History Assessment 




-t 





Primary 

Source Questions 


All Other 
Questions 


Grade 4 


44 (0.5) 


40 (0.4) 


Male 


43 (0.8) 


40 (0.5) 


Female 


44 (0.6) 


40 (0.4) 


White 


47 (0.6) 


44 (0.5) 


Black 


34 (0.8) 


30 (0.5) 


Hispanic 


34(1.3) 


31 (0.9) 



ode 8 


40 (0.4) 


44 (0.3) 


Male 


39 (0.6) 


45 (0.4) 


Female 


41 (0.4) 


44 (0.3) 


White 


43 (0.5) 


48 (0.4) 


Black 


33 (0.8) 


34 (0.5) 


Hispanic 


34 (0.6) 


37 (0.6) 



Grade 12 


35 (0.4) 


43 (0.4) 


Male 


35 (0.4) 


45 (0.4) 


Female 


36 (0.5) 


42 (0.5) 


White 


38 (0.4) 


46 (0.5) 


Black 


27 (0.6) 


32 (0.7) 


Hispanic 


27 (0.7) 


34 (0.6) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 U.S. History Assessment. 
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At grade 4, male and female performance was not significantly different on 
primary source questions or on other questions. Comparisons across gender groups of 
eighth- and twelfth-grade students, however, show patterns of performance on the 
primary source questions that differ somewhat compared to the patterns of performance 
on other questions in the history assessment. At grade 8, female students outperformed 
male students on the primary source questions while their performance on the other U.S. 
history questions was not significantly different from their male counterparts. At 
grade 1 2, male and female students performed similarly on primary source questions 
while male students outperformed female students on the other U.S. history questions. 
The patterns of performance by racial/ethnic subgroups were the same on the primary 
source questions as they were on all other questions in the U.S. history assessment with 
one exception: At grade 8, Hispanic students outperformed Black students on non- 
primary source questions, but their performance on the primary source questions was not 
significantly different. 



2 
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Examples of 1994 NAEP U.S. History 
Primary Source Questions 

Although the common feature of the questions being discussed in this chapter is the use 
of primary source material as a stimulus, the difficulty of each individual question is 
determined by a variety of factors. These factors may include the student’s factual 
historical knowledge, the student’s ability to understand the literary aspects of the primary 
source (such as vocabulary and the use of metaphors), and the depth of analysis and 
interpretation the student is expected to apply to the document’s content. 

In general, one might expect that students would find tasks that ask them to 
provide factual information about a primary source document easier than questions that 
ask them to read and then interpret, analyze, or critique such a document. However, this 
was not necessarily true for all questions. In some cases, questions that required simple 
recall of factual knowledge proved more difficult, apparently because the primary source 
excerpts were simply unknown to the students and contextual clues did not provide 
sufficient help. 
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Recalling Factual Knowledge 

In a number of the primary source questions, students were presented with an excerpt and 
asked to identify the document from which it was extracted, the author of the document, 
or the date or period in which the document was authored. These factual questions appear 
to be quite straightforward. That is, in most cases, the student either had the requisite 
factual knowledge or not. 

Sample U.S. History Question 
Grade 4 — Speech: “ I Have A Dream ” 



Questions 14-15 are about the portion of a speech shown below. 

I have a dream that one day this nation will rise up and live out the 
true meaning of its creed: "We hold these truths to be self-evident; that 
all men are created equal." . . . 

I have a dream that one day on the red hills of Georgia the sons of 
former slaves and the sons of former slaveowners will be able to sit down 
together at the table of brotherhood . . . 

I have a dream that my four little children will one day live in a nation 
where they will not be judged by the color of their skin but by the content 
of their character . . . 

I have a dream that one day . . . little Black boys and Black girls will be 
able to join hands with little White boys and girls and walk together as 
sisters and brothers. 

Reprinted by permission of Joan Daves from I Have a Dream . 

Copyright (c) 1963 by Martin Luther King, Jr. 

14. The speech was given by 
(A) Abraham Lincoln 
CD Gloria Steinem 
<© George Bush 
^ Martin Luther King, Jr. 



The question asks for the name of the speaker from whose speech the excerpt 
was taken. As with other multiple-choice questions, students may have been able to 
eliminate the more unlikely options. However, to answer this question correctly with 
certainty, students needed to know that the excerpt is from a speech given by Martin 
Luther King, Jr. 
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Using the Tools and Resources of Geography and U.S. History 



The student performance data, which are presented in Table 2.4, indicate that the 
“I Have A Dream” speech is a well-recognized speech and so is the orator. The large 
majority of the fourth-grade students in the nation and in different subgroups were able to 
correctly answer this question. Furthermore, on this question Black students 
outperformed Hispanic students, and the performance of Black students was not 
significantly different from the performance of White students. The finding may be 
viewed as commensurate with the importance and relevance of this speech and Martin 
Luther King, Jr. in the history of African Americans. 



Table 2.4 


Percentage Correct for 
Speech: "1 Have a Dream" 


THE N 
REPORT 
CARD 


LATION’S 





Grade 4 87(1.0) 

Male 86(1.4) 

Female 88(1.3) 

White 90(1.1) 

Black 85 (2.3) 

Hispanic 75 (2.9) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 U.S. History Assessment. 
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Sample U.S. History Question 

Grades 4 and 8 — Speech: “A House Divided ” 

The following question, which also depended on students’ recall of factual knowledge, 
was given to both fourth and eighth graders. It is presented here to show how students 
perform differently on questions that pose similar cognitive tasks but require different 
content knowledge. 



Questions 11-12 refer to the statement below. 

A house divided against itself cannot stand. I believe this government 
cannot endure permanently half slave and half free. I do not expect the 
Union to be dissolved — I do not expect the house to fall — but I do expect 
it will cease to be divided. 

11. This statement was made by 
(a) George Washington 
d) Thomas Jefferson 
• Abraham Lincoln 
CD Theodore Roosevelt 
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Using the Tools and Resources of Geography and U.S. History 



To answer this question correctly, students needed to know that the excerpt is 
from a speech given by Abraham Lincoln. Although this question, like the previous one, 
asked students to recall or identify the author of the excerpt, fourth-grade students found 
this question substantially more difficult. As shown in Table 2.5, only 39 percent of all 
fourth graders answered correctly. 





Grade 4 


Grade 8 


Total 


39(1.4) 


55 (1.6) 


Male 


43(1.9) 


56 (2.1) 


Female 


34 (2.0) 


54 (2.1) 


White 


39(1.9) 


59 (2.1) 


Black 


37 (3.2) 


47 (3.3) 


Hispanic 


35 (3.7) 


47 (4.3) 


Standard errors of the estimated percentages appear 


in parentheses. 


SOURCE: National Center for Education Statistics, National Assessment of 


Educational Progress (NAEP), 1994 U.S. History Assessment. 



Most fourth graders have not yet been given formal course work in U.S. history; 
therefore, it is likely that more fourth graders were able to identify Martin Luther 
King, Jr.’s speech because the “I Have A Dream” speech has been referenced more often 
in the media than has Lincoln’s “A House Divided” speech. With regard to subgroup 
differences, although male and female fourth graders’ performance was not significantly 
different on primary source questions overall or on the U.S. history assessment as a 
whole, on the “House Divided” question, male fourth-grade students outperformed 
female fourth-grade students. 

Eighth-grade students performed somewhat better on this question than did 
fourth graders, with 55 percent answering correctly. If students did not specifically 
recognize the orator as Lincoln, the correct answer could have been inferred by 
recognizing the Civil War theme in the speech and identifying Lincoln as the only person 
among the response options who is from the correct historical period. The superior 
performance of eighth-grade students compared to fourth-grade students presumably 
reflects their greater exposure to U.S. history content, and particularly, in this case, to the 
history of the Civil War. 
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Sample U.S. History Question 

Grade 8 — Document: “Declaration of Independence” 

The following question is another example of a question that depended on students’ recall 
of factual knowledge, but, in this case, students were asked to supply the name of the 
primary source document. 



Questions 4-6 refer to the passage below. 

We hold these truths to be self-evident: That all men are created equal; 
that they are endowed by their Creator with certain unalienable rights; that 
among these are life, liberty, and the pursuit of happiness. That, to secure 
these rights, governments are instituted among men, deriving their just powers 
| from the consent of the governed; that, whenever any form of government 
I becomes destructive of these ends, it is the right of the people to alter or to 
1 abolish it, and to institute a new government. 

| —1776 

i 4 . The quotation is from the 
j ® Articles of Confederation 

CD Constitution 

j @ Declaration of Independence 

I CD Missouri Compromise 



'•£2 T r ■rrr- »--t‘ i.T’Trr v s.TTiT'v w.mj'r , ••.«. ■*■*?*- •. ■. r. ■ •: 
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To answer this question correctly, students needed to know that the passage 
came from the Declaration of Independence. The insertion of the date “1776” at the 
bottom of the passage places the document in time and provides an additional contextual 
clue to the identity of the document. The data in Table 2.6 show that 62 percent of eighth- 
grade students knew that the quotation was from the Declaration of Independence. 



' : .so ■ 
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Grade 8 


62(1.4) 


Male 


58(1.9) 


Female 


66 (2.0) 


White 


63(1.7) 


Black 


59 (3.6) 


Hispanic 


57 (3.9) 


Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 


Educational Progress (NAEP), 1994 U.S. History Assessment. 



Of the incorrect responses, the most popular was option B, the Constitution, 
selected by over a fourth of the students. This probably reflects the fact that students are 
familiar with the passage and know that it is from an important United States historical 
document. The Constitution, along with the Declaration of Independence, is much better 
known than either of the other documents named in the response options. 




Students also were presented with questions using primary sources as stimuli and asked 
to recognize the primary source, understand its content, and place it in its historical 
context. In some of these questions, they were asked to interpret the meaning of phrases, 
literary references, or ideas contained in the primary source. In other questions, students 
were asked to describe or explain the context within which the primary source document 
was created. For example, students were presented with songs, poetry, and speeches from 
past historical periods and asked to explain the meaning of selected lines within them. Or, 
students were given passages from documents and asked to explain the circumstances or 
historical context of the documents. Often students were required to use outside 
knowledge to answer the questions posed. 
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Sample U.S. History Question 
Grade 4 — Speech: “ I Have A Dream ” 

In the following question, students were asked to interpret the meaning of a statement 
made by Martin Luther King, Jr. and to explain the political and social circumstances in 
which the speech was written and delivered. To answer this question completely, students 
needed to know and understand the historical context of the speech. 



Questions 14-15 are about the portion of a speech shown below. 

I have a dream that one day this nation will rise up and live out the 
true meaning of its creed: "We hold these truths to be self-evident; that 
all men are created equal." . . . 

I have a dream that one day on the red hills of Georgia the sons of 
former slaves and the sons of former slaveowners will be able to sit down 
together at the table of brotherhood . . . 

I have a dream that my four little children will one day live in a nation 
where they will not be judged by the color of their skin but by the content 
of their character ... 

I have a dream that one day . . . little Black boys and Black girls will be 
able to join hands with little White boys and girls and walk together as 
sisters and brothers. 



Reprinted by permission of Joan Daves from 1 Have a Dream . 
Copyright (c) 1963 by Martin Luther King, Jr. 



15. The speaker dreamed that one day children would not be judged by the 
color of their skin. What did the speaker probably mean by this? 

Explain what was going on in the country that made the speaker’s wishes only 
“dreams.” 




37 



For this short constructed-response question, a “Complete” response correctly 
explained: 1) that not being judged by the color of one’s skin refers to an end to racism or 
the absence of prejudice based on skin color, and 2) that it was a dream for King because 
at that time segregated housing, schools, transportation, and restaurants still existed. A 
“Partial” response correctly explained only one part of the question or attempted both 
explanations but offered responses that were vague or simply repeated what was said in 
the passage. “Inappropriate” responses did not explain what King meant by being judged 
by the color of one’s skin, nor did they explain the circumstances in the country that made 
King’s wishes a dream. The following are examples of actual student responses in each of 
the three response categories. 

Sample “Complete” Response 

15. The speaker dreamed that one day children would not be judged by the 

color of their skin. What did the speaker probably mean by this? J 

/LhJLjjha, /Wn£;. 

Explain what was going on in the country that made the speaker’s wishes only 
“dreams.” 

Jp es rfrjQj UJ4A JL,. — 
rrucfO 

Sample “Partial” Response 

15. The speaker dreamed that one day children would not be judged by the 
color of their skin. What did the speaker probably mean by this? 

DonaE juA^e- y/-,ar5vKi ‘ r\ \ fr-S 

. >\ K ah, i r\& i<jte -VWxT Co ante. 



Explain what was going on in the country that made the speaker’s wishes only 
“dreams.” 

Re-Crtit v e lp i sK <-.& oA u> o-y s tUnOr cortv*- 1 r*xe. 
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Sample “Inappropriate” Response 



I 



I 

I 



15. The speaker dreamed that one day children would not be judged by the 
color of their skin. What did the speaker probably mean by this? 

ON LoWg.-O ( ~To flaad 

>S-Gheec,V\ 



Explain what was going on in the country that made the speaker’s wishes onlv 
“dreams.” 







Js>n*krf^ 



As the information provided in Table 2.7 shows, one-fourth of fourth-grade 
students gave responses that were “Complete.” An additional 45 percent of the responses 
either explained what King meant by being judged by the color of their skin or explained 
the circumstances that made his wishes still a dream. The answers given by 22 percent of 
the students were deemed “Inappropriate,” and 9 percent gave no answer at all. 



) 








Complete 


Partial 


Inappropriate 


Omit 


Grade 4 


25(1.1) 


45(1.2) 


22(1.3) 


9 (0.9) 


Male 


22(1.6) 


45 (2.2) 


20(1.9) 


14(1.6) 


Female 


28(1.7) 


45(1.8) 


23(1.7) 


4 (0.7) 


White 


28(1.4) 


48(1.3) 


18(1.4) 


6 (0.7) 


Black 


18(2.1) 


41 (2.9) 


26 (2.7) 


16(2.6) 


Hispanic 


16(2.6) 


30 (3.6) 


39 (4.8) 


15(3.4) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Proqress (NAEP) 
1 994 U.S. History Assessment. 



A higher percentage of female than male fourth-grade students received a 
Complete” score on this question, although on primary source questions in general and 
on the U.S. history assessment as a whole, male and female students’ performance was 
not significantly different. Also, a higher percentage of White students than Black 
students were able to provide “Complete” responses. However, there was no significant 
difference between the percentages of White and Black students who were able to provide 
“Partial” responses to the question. 








Sample U.S. History Question 

Grade 8 — Document: (( U.S. Constitution, The Fifteenth Amendment ” 
The following question is another example of a task in which students were required to 
read the original source, understand its content, recall factual knowledge, and relate the 
content of the source to its historical context. 



£ortiet| Congress of t|je Unite* states of America; 

|t tfc /fcrrr 

ml hU a to t*j tf a la*), to , - kk «*< k** 
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14. Which group was this amendment designed to help? 

i 

Why was it proposed at this particular time in United States history? 
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To answer this question correctly, students needed to understand that the 
resolution concerns the Fifteenth Amendment to the Constitution; either to know the 
purpose of the Fifteenth Amendment or to discern it from the information presented in the 
primary source document; and to bring in outside knowledge regarding the historical 
context in which the amendment was proposed and passed. The date of the resolution 
provides a clue as to the historical period in which the resolution was proposed. 

For this short constructed-response question, a “Complete” response: 

1) identified former slaves and/or all African American males as people the resolution 
was designed to help, and 2) identified the end of the Civil War, the Radical 
Reconstruction, or the “Black Codes” as the reason for the amendment. A “Partial” 
response either identified who the amendment was designed to help or placed it in its 
proper historical context, but did not do both. An “Inappropriate” response did neither. 
The following are examples of student responses. 

Sample “Complete” Response 



14. Which group was this amendment designed to help? -KTa, 

Why was it proposed at this particular time in United States history? 

WoAllq 

-Urtrt -^TjL^ rQ Ckl £kad iurr^lljCL 





Sample “Partial” Response 

14. Which group was this amendment designed to help? folojC.V -4 
Why was it proposed at this particular time in United States history? 



-Lit — Oino-ttrui/L 

fVtvx-cfLHo. Ck 


-K* 

'ttULe. < AcCf‘t 







O 
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Sample “Inappropriate” Response 



14. Which group was this amendment designed to help? 

Why was it proposed at this particular time in United States history? 

4\\p peftpy . — Ca — 

O.VMXfevr^ w Vo — He 

Q£>\nCj 



The data in Table 2.8 show that eighth-grade students found this question to be 
very difficult. Only a very small percentage of eighth-grade students were able to provide 
responses that were scored as “Complete.” In fact, over half of the students provided 
responses that were scored “Inappropriate.” In addition, over 20 percent of students chose 
not to respond to this question. 







: : V 


Complete Partial Inappropriate Omit 



Grade 8 


5 (0.5) 


17(1.1) 


57(1.6) 


20(1.1) 


Male 


5 (0.9) 


16(1.4) 


55 (2.3) 


25 (2.0) 


Female 


6 (0.9) 


19(1.8) 


60(1.8) 


16(1.2) 


White 


6 (0.7) 


20(1.3) 


57(1.8) 


17(1.3) 


Black 


2 (0.9) 


11 (2.1) 


58 (3.4) 


29 (2.8) 


Hispanic 


3(1.2) 


10(2.0) 


58 (2.4) 


30 (2.5) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 U.S. History Assessment. 



The low performance level is probably a function of both the character of the 
stimulus itself and the substantive content of the question. The stimulus presented could 
have been intimidating for students who were not familiar with actual historical 
documents. Not only is the document handwritten in a script that is difficult to read, but 
the language in which the resolution is written is also formal and perhaps unfamiliar to 
the typical eighth grader. 
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Sample U.S. History Question 

Grades 8 and 12 — Newspaper Report: (( City of Charleston” 

In many cases in the assessment, two or more questions were based on the same stimulus. 
The following is an example of such a case. 



Questions 8-9 refer to the newspaper report below. 

A city of ruins, of desolation, of vacant houses, of widowed women, 
rotting wharves, of deserted warehouses ... acres of pitiful and voiceless 
barrenness — that is Charleston. 

8. The news report was most likely written in 
© 1835 

CD 1845 
CD 1855 
1865 

9. The news report best supports which statement? 

<© Cities on the coast saw the worst of the fighting in the Civil War. 

During the Civil War urban areas in the South suffered. 

CD The destruction of cities had little effect on the progress of the 
Civil War. 

CD The Civil War had little effect on city life in the North. 



In the first question, students were asked to identify the date when Charleston 
may have been so described. The options focused students on the period of and prior to 
the Civil War. Students needed to know the year the Civil War ended and also that 
Charleston was burned during the last stages of the war. 



As evidenced by the data presented in Table 2.9, less than a third of eighth-grade 
students were able to answer this question correctly. Apparently, many eighth-grade 
students either did not understand that the description was of the devastation that 
occurred to Charleston as a result of the Civil War, or they simply did not know the date 
of the end of the Civil War. 





Grade 8 


Grade 12 


Total 


32 (1.4) 


41 (1.3) 


Male 


34(1.8) 


45 (2.0) 


Female 


30 (2.0) 


36(1.8) 


White 


34(1.7) 


43(1.6) 


Black 


31 (3.1) 


37 (3.2) 


Hispanic 


30 (2.4) 


32 (3.2) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 


(NAEP), 1994 U.S. History Assessment. 







A higher percentage of twelfth graders answered this question correctly; 
however, this group still represented less than half of the students. Further, although 
twelfth-grade female and male students performed similarly overall on the primary source 
questions, on this question, male students performed better than female students. 
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The second question covered the historical context of the newspaper report on 
Charleston. It attempts to evaluate the students’ ability to understand rules of inference. It 
asks students to use the information in the news report to draw a supportable conclusion. 
Students needed to recognize that the news report only supports option B. Option A is 
incorrect because, although Charleston is on the coast and is portrayed in the newspaper 
report as devastated, the quotation does not provide evidence that coastal fighting was 
worse than that experienced in other areas. Similarly, for option C, although the news 
report is about the destruction of an urban area, there is no reference to how this may have 
affected the progress of the war. And, for option D, there is simply no reference to city 
life in the North. Student performance information is presented in Table 2.10. 

















Grade 8 


Grade 12 




Total 

Male 

Female 

White 

Black 

Hispanic 


47(1.5) 
45 (2.0) 
50(1.9) 
50(1.9) 
43 (3.0) 
40 (3.0) 


61 (1.6) 

61 (2.0) 
62(2.1) 
63 (2.0) 
58 (4.1) 
51 (4.0) 





Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 U.S. History Assessment. 



Here is an example of the phenomenon mentioned earlier: that students at times 
found certain factual recall questions more difficult than higher-level inferential skills 
questions. At both grades, a higher percentage of students were able to answer this 
question correctly than were able to answer the question regarding the time in which 
the news report was written. Almost half of the eighth-grade students, and more 
than 60 percent of twelfth-grade students, answered this question correctly. One 
possible reason that the inferential question was easier than the actual recall question 
at both grades is that in answering the inferential question, students could draw 
on their deductive reasoning skills rather than simply their ability to remember 
historical information. 
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A substantial number of primary source questions, particularly in the twelfth-grade U.S. 
history assessment, asked students to compare and contrast differing points of view on a 
particular issue. In some of these questions, students were presented with two or more 
primary sources and were required to compare and contrast the positions in those sources 
In other questions, students were asked to use outside knowledge to present an opposing 
view to the one referenced in the primary source(s). To fully respond to most of these 
questions, students needed outside knowledge about historical themes, ideas, and time 
periods. 

Sample U.S. History Question 

Grade 12 — Document: “U.S. Constitution: State Representation ” 



Article I. Section 2. 

I 

| 

“Representatives . . . shall be apportioned among the several states 
J which may be included within this Union, according to their respective 
| numbers . . .” 

J Article I. Section 3. 

“The senate of the United States shall be composed of two senators from 
each state . . .” 

I 

3. An important debate between large states and small states led to the writing 
of these portions of the Constitution. Identify the issue being debated. 

What position on this issue did large states hold, and why did they hold it? 

What position on this issue did small states hold, and why did they hold it? 



This was a rather straightforward question in that it provided students with the 
primary source, identified the source as the Constitution, and indicated that the passage 
was related to an important debate between large and small states. However, having been 
provided with all this information, students still needed to bring in outside knowledge to 
fully respond. They needed prior knowledge about the historical debate on state 
representation in the U.S. Congress, the positions of large and small states, and the fact 
that these sections of Article I of the Constitution regarding membership of the House of 
Representatives and Senate were adopted as a solution to that debate. 




As/#//.* i/c ' 'tit: !\ c..*! 
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For this extended constructed-response question, a “Complete” response: 

1) identified the issue being debated as state representation in the U.S. Congress; 

2) indicated that the position of large states was that representation should be based on 
population, which would increase the power of large states and, it might be argued, be 
more democratic; and 3) explained that the position of small states was that 
representation should be equal for each state, which would increase the power of small 
states and, it might could be argued, be more consistent with the concept of federalism. 
An “Essential” response correctly answered two parts of the question, but not the third. A 
“Partial” response correctly answered one part of the question but not the other two. An 
“Inappropriate” response did not accurately answer any part of the question. Below are 
examples of actual student responses to this question. 

Sample “Complete” Response 



3. 



An important debate between large states and small states led to the writing 
of these portions of the Constitution. Identify the issue being debated. 






What position on this issue did large states hold, and why did they hold it? 

■TTA-l. 

j£Ll2^A 

What position on this issue did small states hold, and why did they hold it? 

jj£ r a^u^JL-. 7L >.AaX^o/ TfcLx 

ytzLi^YK W\ 
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Sample “Essential” Response 

3. An important debate between large states and small states led to the writing 
of these portions of the Constitution. Identify the issue being debated. 

iArcy=> Rirrbps -the. larger -ting, state -Vine, rtor e 

The smaller states ary^d -Hnis dan se. 
4-hr. wonted erftja.\ tepreserHodioA u.nr-e\a.4<ed to sn ^ 

What position on this issue did large states hold, and why did they hold it? 
TV\fij vonnhfd nnOfg fg.pf e^ervtcx-\-i 

flnetj VEliejgd wij Cpufci p\£rrujr\ +W2. 

srvnJiar . 

What position on this issue did small states hold, and why did they hold it? 

~Th£t^ \xx\n\e.d <€ pr e^serii^Jnarx ncrV 

Mo te> 

Sample “Partial” Response 

3. An important debate between large states and small states led to the writing 
of these portions of the Constitution. Identify the issue being debated. 

u£ jJU. -irrxt&d 

• QAeJh 

< -J_ U2a^d t JtJu . 

What position on this issue did large states hold, and why did they hold it? 

— Adlct tsnn^ ni 

. UU^LAJL ^i^~C.CXjU^L — Suuy ^yyy\ en g ( ^ *<y ^ 

What position on this issue did small states hold, and why did they hold it? 
g^»>T g &£_ — &&£&. — <-^C^enj<y-esi 

/& jZdAfOjA* 
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Sample “Inappropriate” Response 



3. An important debate between large states and small states led to the writing 
of these portions of the Constitution. Identify the issue being debated. 

Tlia diJbaJtpAs 

ejbdtodCve. 5 sIapuM* oam^qaaJ ^ 

tLu va ^ 



What position on this issue did large states hold, and why did they hold it? 

TVte poftifcafry oajq bU&£ 

kWe <Q&AUltQ gVjQXXs bO Corv^po^/L <o^_ t/KfO 
^uOAM' QXUUa/ 

What position on this issue did small states hold, and why did they hold it? 



According to the student response data presented in Table 2.1 1, 17 percent of 
students were able to answer each of the three parts of the question appropriately, and an 
additional 15 percent were able to provide appropriate responses to two of three parts of 
the question. These 15 percent of students appeared to understand that the basis of the 
debate was representation in Congress. They had more difficulty in expressing the reasons 
for the positions taken by either the large or small states. The fact that the question 
identified the parties to the debate (large versus small states) may have helped students 
provide more complete answers to this question than they would have been able to 
provide otherwise. 





Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 12 


17(1.2) 


15(1.0) 


24(1.2) 


30(1.3) 


14(1.1) 


Male 


17(1.6) 


15(1.4) 


25(1.8) 


28(1.7) 


15(1.6) 


Female 


17(1.9) 


15(1.2) 


23(1.7) 


32(1.9) 


13(1.2) 


White 


19(1.4) 


16(1.4) 


27(1.6) 


27(1.8) 


11 (1.3) 


Black 


10(2.2) 


6(1.6) 


22 (2.6) 


42 (2.9) 


21 (2.1) 


Hispanic 


2 (0.9) 


18(2.5) 


12(2.2) 


42 (3.3) 


27 (3.3) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 U.S. History Assessment. 
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Sample U.S. History Question 

Grade 12 — Documents: “How Crash Affected People ” 

For the following questions, students were presented with original documents describing 
Black Thursday, the stock market crash of October 24, 1929. The first document was a 
newspaper report while the other two documents offered different personal accounts of 
the day. These materials and tasks were presented as part of a 50-minute assessment unit 
that focused on the Great Depression. 

Students were asked in two different questions to compare and contrast various 
aspects of the documents and their contents. Although it helped for students to know 
about and understand the circumstances of the crash and its impacts on society, to answer 
the questions posed, it was really only necessary to have rudimentary information about 
the crash. It was, however, essential for students to be able to read and understand the 
information in the documents and to be able to compare and contrast the information 
provided. 
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The first three documents { Documents A-Cfare about Black Thursday, the stock market 
crash of October 24, 1929. 

Document A - The New York Times. October OS 1000 


THE MOST DISASTROUS DECLINE in the biggest and broadest stock market of 
history rocked the financial district yesterday. In the very midst of the collapse five 
of the country’s most influential bankers hurried to the office of J. P. Morgan & Co., 
and after a brief conference gave out word that they believe the foundations of the 
market to be sound, that the market smash has been caused by technical rather than 
fundamental considerations, and that many sound stocks are selling too low. 

The decline carried down with it speculators, big and little, in every part of the 
country, wiping out thousands of accounts. It is probable that if the stockholders of 
the country’s foremost corporations had not been calmed by the attitude of leading 
bankers and the subsequent rally, the business of the country would have been seri- 
ously affected. Doubtless business will feel the effects of the drastic stock shake-out, 
and this is expected to hit the luxuries most severely. 


elf 

he 


Document B — In 1929 Gordon Parks — later a world-famous photographer — was sixteen 
years old, living alone and attending high school in St. Paul, Minnesota. To support himsi 
he worked part-time as a bellboy at an exclusive club. Here he describes the impact of t 
stock market crash on his life. 


By Thursday the entire world knew. “MARKET CRASHES — PANIC HITS 
NATION!” one headline blared. The newspapers were full of it, and I read every- 
thing I could get my hands on, gathering in the full meaning of such terms as Black 
Thursday, deflation, and depression. I couldn’t imagine such financial disaster 
touching my small world; it surely concerned only the rich. But by the first week of 
November I too knew differently; along with millions of others across the nation, I 
was without a job. All that next week I searched for any kind of work that would 
prevent my leaving school. Again it was, “We’re firing, not hiring.” “Sorry, sonny, 
nothing doing here.” Finally, on the seventh of November I went to school and 
cleaned out my locker, knowing it was impossible to stay on. A piercing chill was in 
the air as I walked back to the rooming house. The hawk had come. I could already 
feel his wings shadowing me. 


Document C — Arthur Robertson, a stockbroker and hanker remember ? the r^t^b 




October 1929, yeah. A frenzy. I must have gotten calls from a dozen and a half 
friends who were desperate. In each case, there was no sense in loaning them the 
money that they would give the broker. Tomorrow they’d be worse off than yester- 
day. Suicides, left and right, made a terrific impression on me, of course. People I 
knew. 

One of my friends said to me, “If things keep on as they are, we’ll all have to go 
begging.” I asked, “Who from?” 

Many brokers did not lose money. They made fortunes on commissions while their 
customers went broke. The only brokers that got hurt badly were those that gambled 
on their own — or failed to sell out in time customers’ accounts that were underwa- 
ter. Of course, the brokerage business fell off badly, and practically all pulled in their 
belts, closed down offices, and threw people out of work. 





1. Documents A and C contain different information about the stock market 
crash. How is the information in the two documents different? 



2. The crash affected people from different economic backgrounds in differ- 
ent ways. Compare how Gordon Parks and Arthur Robertson were affected 
by the crash. 
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The first question is an extended constructed-response question that focuses on 
the newspaper report and the personal account by the stockbroker. To receive a 
“Complete” score, a response to this question needed to show an understanding of the 
difference between a newspaper account and a personal account or between a 
contemporary account and a remembrance. Credited responses included information such 
as that the news account reported on the size of the crash and the reaction of the business 
community, but tended to underestimate the importance of the crash to individuals. The 
personal account included information on the effect the crash had on business people and 
the financial system, and it tended to describe the crash in more human terms. An 
“Essential” response showed an understanding of both types of documents but was vague 
or inaccurate about the differences between the perspectives in both accounts. A “Partial” 
response showed an understanding of only one of the documents and, although it may 
have identified a difference between the documents, the difference was a relatively 
unimportant one. An “Inappropriate” response did not correctly identify any of the ways 
in which the two accounts differed. Examples of student responses in each of the 
response categories are presented below. 

Sample “Complete” Response 

1. Documents A and C contain different information about the stock market 
crash. How is the information in the two documents different? 




. Ai r (34^ ~Lo "fi -lVYl. ■ 
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Sample “Essential” Response 



1. Documents A and C contain different information about the stock market 
crash. How is the information in the two documents different? 

Vs rExO-fia/CtCt 'l-a V , 

h>P ^ Wq. -{-TiJPj-i rploQuJt 

ciDQ Vv\p^ 'wyirV^V- Wju-V po ^v^ilg- 

• klW>r-k c^J-Q^ W^jufVtA^ 



Sample “Partial” Response 



1. Documents A and C contain different information about the stock market 
crash. How is the information in the two documents different? 

c\Docu/m//*i g cui fyeoQ TlLai Ipt L n^ivud la ipssdl^ 

^Qj^SLuzdcJo^z. uiLUdbcc^ tXAfLuzzm' 

JJLL/1A o l rJ mnnJt ±AulS*Cq 4r> ctAJU LiA\.^Ti 
Q _ i7 0 V 

sf) cUuAty. £ JddAAK^ O' Juh 

iZAXL* 




Sample “Inappropriate” Response 



1. Documents A and C contain different information about the stock market 
crash. How is the information in the two documents different? 






u> 




Uf>6CUrrusCt£L 



L-CLDjx/xl C, ira^ nru qj lA^dbcul, i 
QjpCL* L^yconQ. 
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The student response data presented in Table 2.12 show that “Complete” 
responses were rare. Very few students were able to successfully articulate the salient 
distinguishing features of the documents (in terms of both content and tone). Over a third 
of the responses were rated “Essential,” an indication that substantially more students 
understood that the two accounts were different, although it was difficult for them to 
express the differences clearly. 



Table 2.12 


Score Percentages for Documents: 

"How Crash Affected People"— Question 1 


THE NATION’S 
REPORT tapH 
CARD “ 


Complete Essential Partial Inappropriate Omit 



Grade 12 


1 (0.3) 


37(1.4) 


35 (1.4) 


22(1.1) 


5 (0.8) 


Male 


1 (0.5) 


33(1.6) 


36 (2.0) 


24(1.3) 


6(1.2) 


Female 


1 (0.3) 


41 (2.1) 


33(1.7) 


21 (1.7) 


5 (0.8) 


White 


1 (0.4) 


40(1.7) 


35(1.7) 


20(1.3) 


4 (0.9) 


Black 


1 (0.5) 


29 (2.8) 


34 (2.8) 


28 (2.7) 


9(2.1) 


Hispanic 


0 (0.0) 


30 (4.4) 


33 (4.1) 


26 (3.7) 


10(3.2) 



Standard errors of the estimated percentages appear in parentheses. _ .. . A _ n . 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAtP), 
1 994 U.S. History Assessment. 



The second question is a short constructed-response question. It focused on the 
two personal accounts of the crash and asked students to do a close comparative reading 
of these accounts. A response scored as “Complete” reflected an understanding of the 
content of both passages and presented appropriate evidence of how both authors were 
affected by the crash. “Complete” responses included information such as Parks was 
poor, worked at a country club, got laid off, had to drop out of school, was on his own, 
and could not find work. Robertson, on the other hand, was affluent and probably not 
badly hurt at first, did not exactly describe how he was financially affected, 
watched friends suffer, and had to refuse their requests for loans. A “Partial” response 
showed an understanding of one of the documents but not both, or showed only a vague 
or superficial understanding of the content of both. “Inappropriate” responses did not 
indicate correctly that the two men were affected differently by the crash or provided only 
incorrect or irrelevant differences. Examples of student responses are provided below. 



Using the Tools and Resources of Geography and U.S. History 



54 



45 



Sample “Complete” Response 



2. The crash affected people from different economic backgrounds in differ- 
ent ways. Compare how Gordon Parks and Arthur Robertson were affected 
by the crash. 

Gro(Jo\ 'tkte Wife derriMA til 4o cty oul J w jj, 
job . i Hirt +*& no /'Cult fiioc-feuiaj -tx Ki^v P\\rWjJT Pntv - 
_ifl5er\ rftwa in kxVf. Wt of-ft off- t** fiQrsk 



J-i cxrmrv, g>>ons, t 

M, bul «!&✓ Hra 



Ki*a lot/S if\<SS €ve.i 



Wakgyy. boStr^sS^QS ff. 






Sample “Partial” Response 



2. The crash affected people from different economic backgrounds in differ- 
ent ways. Compare how Gordon Parks and Arthur Robertson were affected 
by the crash. __ 

Jjrrvt<rw\ V4 a fe^avi-lflcd 

JrML ^YXjd- ftm/iKfri to mirT-t- 



1 



-Vx Xu flft in 

cmxn Cl/lDVUA ^JU A^/n . 

^QTu /liL I'tOO) A guYlPtf/ ^ 

i I a /1 _\-L /] /vr\/» ^ 



CSUii^xH liSOQ. pr\iTnftij 



Sample “Inappropriate” Response 



2. The crash affected people from different economic backgrounds in differ- 
ent ways. Compare how Gordon Parks and Arthur Robertson were affected 
by the crash. 

rJ)+*U 

Jt. AiiJ., >iuI y>>Awo(!L CbdtLhs 

<XJ** A A 
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According to the student performance data presented in Table 2.13, 13 percent 
of the students were able to understand and describe important differences in the two 
personal accounts of the crash. 



Table 2.13 


Score Percentages for Documents: 


THE N 
REPORT 
CARD 


lATlON’S 

rsaep 




"How Crash Affected People" —Question 2 




-S: 





Complete 


Partial 


Inappropriate 


Omit 


Grade 1 2 


13(0.8) 


58(1.3) 


24 (0.9) 


6(0.7) 


Male 


11 (1-2) 


55 (2.1) 


27(1.7) 


8(1.0) 


Female 


14(1.3) 


60 (1 .6) 


21 (1.5) 


5 (0.8) 


White 


14(0.9) 


61 (1.6) 


21 (1.2) 


5 (0.8) 


Black 


9(1.6) 


46 (3.5) 


33 (2.8) 


12(2.0) 


Hispanic 


7(2.0) 


50 (4.5) 


28 (2.9) 


15(3.8) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 


1 994 U.S. History Assessment. 









Student performance on these two questions shows that students may have 
difficulty responding fully to questions that ask them to make distinctions among 
documents and the kinds of information and perspectives they offer. Students were able 
to discuss simple content differences, but they performed less well when asked to explain 
why these differences exist or to describe the differences in more complex terms. 
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Summary 

Student performance on questions based on primary source documents varied widely. 
Precise statistical explanations of this variation are beyond the scope of this report. 
However, an examination of the assessment questions suggests that the differing 
difficulty of the primary source exercises seem related to three factors: the factual 
knowledge that students were required to bring to the questions, the format of primary 
source documents presented, and the level of cognitive skills required to respond fully. 
When the historical content was known or the primary source document was relatively 
straightforward in presentation of information, higher proportions of students were able to 
explain contextual circumstances and were able to compare and contrast differing points 
of view. However, when the format and language of the primary source documents were 
less familiar, the task became more difficult. For example, although eighth-grade students 
have probably heard of the Fifteenth Amendment and the circumstances surrounding it, 
when presented with the original resolution proposing it to states, only a small percentage 
of students could identify the groups that the amendment was designed to help or explain 
why it was proposed at the time. 

Performance also depended on the complexity of analyses students were 
required to undertake when answering questions. For example, when asked to compare 
and contrast the contents of two documents as in the question on the stock market crash, 
students were generally able to discuss superficial differences with ease. However, they 
were less able to perform in-depth analyses of the contents and relate these to historical 
contexts. 



Most history education and curriculum specialists believe that primary source 
documents should be an essential part of history instruction. The use of such documents 
is viewed by professionals as important for a number of reasons. Primary sources are an 
effective means of imparting content knowledge. Learning to interpret such documents 
helps students to develop the critical-reasoning skills valued by historians. Finally, 
primary sources are also the tools with which professional historians work; if students are 
to develop historical “habits of mind” they must learn to understand and interpret written 
materials from the past. However, while there is broad agreement on the importance of 
primary documents, they have only found a limited place in elementary, middle, and 
secondary classrooms. And not surprisingly, given this finding, students still found it 
quite difficult to perform the interpretive tasks involving primary source documents that 
were presented to them as part of the 1994 NAEP U.S. history survey. 
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Chapter 3 



Working with Maps 

A tremendous amount of information can be presented on maps. Maps can show locations 
of cities, countries, and continents; the topography of different regions; distributions of 
natural resources and populations; weather and climatic conditions; transportation routes 
such as highways and city rapid transit lines; and patterns of migration and settlement. 
Maps and globes are the primary tools of geography. They help in the visualization of 
space and can be used to communicate complex social and physical data. They are a 
resource for information about the world, past and present. The ability to access 
information from maps is a valuable, even essential, skill. It is, therefore, important for 
students to learn about maps and how to use them. 

To be map literate, students must learn to recognize and understand map-making 
conventions. First, cartographers use symbols on maps to indicate locations of natural and 
synthetic features. For example, on a highway map, a red line may be used as the symbol 
for a major interstate while a thin black line indicates a two-lane highway. Pictorial 
representations on maps usually suggest the features they represent — such as a squiggle 
for a river, an outline of an airplane for an airport, or a collection of tree outlines for a 
forest. In addition to these types of symbols, map makers also use colors or shadings to 
indicate features such as political boundaries, elevation, and natural geographical 
features. To use a map effectively, therefore, it is important for students to understand the 
use of symbols and, of course, the essential role of the key that defines the symbols. 

A second map-making convention is the use of a scale. Maps are not drawn to 
actual size. The scale to which a map is drawn generally is indicated on the map either 
with words, such as “1 inch equals 100 miles,” or graphically, with a line marked off into 
the relevant units. Understanding the use of scale is important in interpreting the 
information being presented on a map. 

A third convention is the use of directional aids, including parallels of latitudes, 
meridians of longitudes, and compass directions. It is important for students to understand 
these devices because they are fundamental tools for referencing the location of places in 
the world and for guiding movement around the world. 
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Classroom Instruction 



As part of the NAEP 1994 geography assessment, students in grades 4, 8, and 12 and 
teachers of fourth-grade students were asked to provide information about classroom 
practices related to the study and use of maps and globes. 1 In particular, they were asked 
about the general use of maps and globes, the measuring of distances on maps, and the 
use of maps to understand latitude and longitude. Tables 3.1 and 3.2 present teachers’ and 
students’ reports on these classroom practices. 




Teachers of the eighth-grade students were also surveyed about instructional activities, but an apparent 
misunderstanding during the administration of the survey resulted in serious data limitations; therefore, 
their responses are not reported. See the NAEP 1994 geography report card for a more detailed 
explanation. Teachers of twelfth-grade students were not surveyed because twelfth-grade students are not 
necessarily enrolled in classes that teach geography. 
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Table 3.2 



Percentage off Students Reporting on 
Frequency off Practices in Geography 
Instruction by Grade Level 



THE NATION’S 
REPORT 



CARD 



raep 



M: 



Grade 4 



Grade 8 



Grade 12 



Use of Maps and Globes 

Almost Every Day 
Once or Twice a Week 
Once or Twice a Month 
Never 



18(0.9) 
32 (0.8) 
23 (0.8) 
27(1.0) 



9 (0.6) 
30 (0.9) 
33 (0.9) 
28(1.0) 



7(0.4) 
22 (0.7) 
31 (0.7) 
40 (0.9) 



Use of Maps to Measure Distances 

Almost Every Day 1 0 (0.6) 

Once or Twice a Week 28 (0.9) 

Once or Twice a Month 27 (0.8) 

Never 35(1.2) 



4 (0.4) 
21 ( 0 . 8 ) 

37 (0.7) 

38 (0.9) 



3 (0.3) 
15(0.5) 
27 (0.7) 
56 (0.9) 



Use of Maps to Understand Latitude and Longitude 

Almost Every Day 1 4 (0.8) 7 (0.4) 

Once or Twice a Week 26 (0.8) 20 (0.9) 

Once or Twice a Month 28(0.7) 38(0.9) 

Never 32(1.1) 35(1.3) 



4(0.2) 
12 ( 0 . 6 ) 
25 (0.7) 
59 (0.9) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 



According to teachers of the fourth-grade students, maps and globes were used 
in nearly all classrooms, with the modal response indicating use once or twice a week. 

The specific map reading skills of measuring distance and using latitude and longitude 
were also reported for the majority of classrooms, although the reported frequency of use 
was lower (the modal response was “once or twice a month”). Fourth-grade students and 
their teachers differed somewhat in their perceptions of the frequency of these practices in 
their classes. For example, teachers of 83 percent of fourth graders reported using maps 
and globes at least once or twice a week while only about half of fourth graders indicated 
such frequency. On the other hand, fourth-grade students reported higher frequencies of 
practices such as measuring distances on maps and use of latitude and longitude than did 
their teachers. 

The majority of eighth graders also reported using maps and globes, measuring 
distances, and using latitude and longitude — with approximately two-thirds indicating 
that these tended to be used at least once or twice a month. At the twelfth grade, however, 
while many students still reported using these tools and practices at least monthly, 
substantial proportions reported no such regular use. A possible reason for this apparent 
decline in use among upper-level students is the diversification of curriculum in high 
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school. Depending upon their chosen courses, twelfth-grade students may be less likely to 
be in classes that utilize maps or globes with any regularity. However, most twelfth-grade 
students have been taught to use and understand maps in earlier grades. 



Map Questions in the 

1994 NAEP Geography Assessment 

This chapter focuses exclusively on map questions in the 1994 NAEP geography 
assessment. Although both the geography and U.S. history assessments included 
questions that used maps as stimuli, maps played an extremely prominent role in the 
geography assessment. In fact, not counting questions in the atlas block (which are being 
treated separately for purposes of this report), 44 percent of the questions in the 
geography assessment across grades 4, 8, and 12 involved the use of maps. 

Students were presented with a wide variety of maps in the 1994 geography 
assessment, including the everyday versions of world maps and many specialized maps, 
such as maps of public transportation routes, city streets, weather patterns, natural 
resources, and population densities. Questions that used maps as stimuli measured a 
variety of knowledge and skills, including the following: 

• Map literacy, or an understanding of the syntax and conventions of maps. 

This includes knowledge about types of map projections; an understanding of 
conventional components of maps, such as contour lines, lines of latitude and 
longitude, scales, and compass direction; and the ability to judge the accuracy 
and appropriateness of map projections for different purposes. 

• The use of maps to locate places, such as continents, countries, and cities; to 
identify natural features, such as mountain ranges, oceans, and deserts; and to 
recognize geographical routes of importance, such as ocean currents and 
migration routes. 

• The use of maps to obtain and integrate information, such as locations of 
places, weather patterns, population densities, and elevations of natural 
geographical features. 
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• The use of map information and outside information to explain real-world 
conditions and issues, such as diffusion of modern-day religions or political 
and economic conflicts. 

• The ability to create maps from textual descriptions using accepted map 
conventions. 

In many of the map questions, students were asked to use a combination of these 
skills in order to respond correctly. For example, a number of questions asked students to 
locate a place or a natural feature on a map and then explain how its location affected 
certain historical or geographical events or conditions. In the 1994 NAEP geography 
assessment, the total number of questions that required students to use maps was 38 at 
fourth grade, 44 at eighth grade, and 48 at twelfth grade. 
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Student Performance on the 
1994 Geography Map Questions 

As the data presented in Table 3.3 show, the difficulty of map questions compared to all 
other questions differed by grade level. Eighth-grade map questions were similar in 
difficulty to other questions in the eighth-grade assessment. But at the fourth- and 
twelfth-grade level, the map questions were easier on average than the other questions. 
At each grade level, however, the patterns of student performance across gender and 
racial/ethnic subgroups were about the same for map questions as they were for all other 
questions in the geography assessment. 



Table 3.3 


Average Item Score 

1994 NAEP Geography Assessment 


THE h 
REPORT 
CARD 


lATION’S 

-d 









Map Questions 


All Other 
Questions 


Grade 4 


48 (0.6) 


45 (0.5) 


Male 


50 (0.8) 


46 (0.6) 


Female 


46 (0.7) 


44 (0.6) 


White 


53 (0.8) 


49 (0.7) 


Black 


30(1.0) 


32 (0.7) 


Hispanic 


38(1.2) 


37(1.1) 


Grade 8 


49 (0.4) 


49 (0.3) 


Male 


51 (0.5) 


50 (0.4) 


Female 


48 (0.5) 


48 (0.4) 


White 


54 (0.5) 


53 (0.4) 


Black 


34 (0.7) 


35 (0.8) 


Hispanic 


40(1.2) 


39 (0.8) 


Grade 12 


55 (0.5) 


47 (0.5) 


Male 


57 (0.5) 


49 (0.6) 


Female 


52 (0.5) 


46 (0.5) 


White 


59 (0.5) 


50 (0.5) 


Black 


38 (0.9) 


34 (0.7) 


Hispanic 


45(1.0) 


38 (0.9) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 Geography Assessment. 
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Examples of 1994 NAEP Geography Map Questions 

Examples of actual assessment questions from the 1994 NAEP geography assessment 
are presented below. The examples are not intended to be representative of all map 
questions. They do, however, show the variety of maps students were asked to read 
and the variety of skills students were asked to employ to correctly respond to these 
questions. 

Map Literacy 

In most of the questions that used maps as stimuli, students were assumed to have 
knowledge about the use of map conventions, including symbols, scales, and compass 
directions; the meaning and purposes of latitude and longitude; or the differences among 
map projections. However, the 1994 geography assessment also included questions that 
directly assessed students’ map literacy. For example, students were asked the names of 
such map features as contour lines, the equator, lines of latitude, and the Tropic of 
Cancer. Twelfth-grade students were also asked to judge the appropriateness of map 
projections — for example, the Mercator and Azimuthal projections — for specific 
purposes, such as guiding a pilot on a flight from North America to Europe over the 
north pole. 

Sample Geography Question 

Grade 4 — World Map: Identify Poles and Equator 

The following is a relatively straightforward question asking students to locate three 

important features of a world map. It tested factual knowledge of map conventions. 




11 . On the map above, write the names of the North Pole, the South Pole, and the 
equator in the correct location. I 
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The question required a short constructed-response. Specifically, to receive a 
score of “Complete,” a student had to have correctly labeled the North Pole, South Pole, 
and equator on the map. A “Partial” response correctly identified only one or two of the 
features. An “Inappropriate” response did not correctly locate any of the three features. 
The following are examples of actual student responses. 



Sample “Complete” Response 




Sample “Partial” Response 
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Sample “Inappropriate” Response 




Student performance on this question is presented in Table 3.4. More than half 
of all fourth graders were able to identify both poles and the equator correctly, and about 
a quarter more were able to identify at least one of the three features. For responses 
scored as “Complete,” the differences in performance among racial/ethnic groups were in 
the same direction as they were for geography questions in general. However, the 
differences between White students and Black students and between White students and 
Hispanic students were especially large on this item. In addition, there were also 
differences in the percentages of students of different racial/ethnic groups who did not 
attempt to answer this question. Thirty-one percent of Black students and 21 percent of 
Hispanic students, compared to 8 percent of White students, omitted this question. 





Score Percentages for World Map: 


THE NATION’S 


Table 3.4 


REPORT 

CARD 


rasp 




Identify Poles and Equator 











Complete 


Partial 


Inappropriate 


Omit 


Grade 4 


54(1.5) 


26(1.2) 


7(0.7) 


13 (0.8) 


Male 


54 (2.0) 


25 (1 .5) 


6 (0.8) 


15(1.2) 


Female 


54 (2.4) 


27(1.8) 


8(1.4) 


10(1.1) 


White 


62(1.6) 


25(1.3) 


6(1.0) 


8 (0.8) 


Black 


28 (3.5) 


29 (3.2) 


12(1.6) 


31 (3.2) 


Hispanic 


41 (3.9) 


28 (3.7) 


10(2.1) 


21 (3.9) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Sample Geography Question 

Grade 8 — Map: Identify Parallels of Latitude 

Like the preceding question, the following question asked students to identify a feature of 
a map. The map, however, was more complex and reflected an expectation that eighth 
graders would have greater knowledge of maps than fourth graders. 




17. Latitude on this map is represented by 
9 circles 
(b) shaded areas 
(D straight lines 
CD convergent lines 



BEST COPY AVAILABLE 




58 



Using the Tools and Resources of Geography and U.S. History 





In order to answer this question correctly, students needed to know the map 
convention of latitude and recognize that, on this map projection, latitude is represented 
by concentric circles. Student performance information is presented in Table 3.5. 



Table 3.5 


Percentage Correct for Map: 


THE N 
REPORT 
CARO 


[ATION'S 

rasp 




Identify Parallels of Latitude 




=4 



Grade 8 


48(1.4) 


Male 


47(2.1) 


Female 


49 (2.1) 


White 


52(1.5) 


Black 


33 (3.5) 


Hispanic 


43 (4.1) 


Standard errors of the estimated percer 


itages appear in parentheses. 


SOURCE: National Center for Education Statistics, National Assessment of Educational 


Progress (NAEP), 1 994 Geography Assessment. 



Nearly half of all eighth graders in the nation appeared to know that latitude is 
represented by the parallel lines that circle the earth. The most frequently chosen incorrect 
answer was option C, straight lines; 29 percent of students selected this option. It is 
possible that these students have some knowledge about latitudes and longitudes, but may 
have confused the two concepts. 

Locating Places and Natural Geographical Features 

Many of the questions on the 1994 geography assessment required a demonstration of 
factual knowledge of places and natural geographical features. Students were asked to 
identify continents, countries, mountain ranges, rivers, oceans, and deserts. There were a 
variety of ways the assessment attempted to have students show this content knowledge. 
Some of these ways are illustrated in the questions presented below. 
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Sample Geography Question 

Grade 4 — World Map: Identify Continents 

The stimulus for the following question was a map of the world with none of the 
locations or natural geographic features identified by name. Students were asked to locate 
four specific continents and color each a designated color; participants were provided 
with colored pencils for this purpose. 



Questions 1-4 are based on the following map. 



THE WORLD 




1. Use your colored pencils. On the map above: 

- color Australia blue 

- color South America yellow 

- color North America green 

- color Asia red 



A “Complete” response to this extended constructed-response question correctly 
located all four of the continents: Australia, South America, North America, and Asia. 

An “Essential” response located three of the four continents correctly. A “Partial” 
response located only one or two of the four continents, and an “Inappropriate’ ’ response 
located none of the four continents correctly. A continent was considered incorrectly 
located if the colored-in area included substantial areas that were definitely not part of the 
continent. 
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Student performance information is presented in Table 3.6. Just over a tenth of 
the students were able to correctly locate all four continents: Australia, South America, 
North America, and Asia. However, half of fourth graders were able to identify at least 
three of the four continents. 



Table 3.6 


Score Percentages for World Map: 
Identify Continents 


THE N 
REPORT 
CARD 


lATION’S 

rasp 

-S’ 


Complete Essential Partial Inappropriate Omit 



Grade 4 


11 (l.D 


40(1.4) 


33(1.3) 


12(1.1) 


5 (0.5) 


Male 


11 (13) 


40 (1 .8) 


35(1.8) 


9(1.3) 


5 (0.7) 


Female 


10(1.5) 


39(1.9) 


31 (1.9) 


16(1.5) 


4 (0.8) 


White 


13(1.5) 


44(1.6) 


32(1.7) 


9(1.3) 


2(0.5) 


Black 


3(1.3) 


28 (2.5) 


32 (3.1) 


24 (2.6) 


13(2.1) 


Hispanic 


4(1.5) 


33 (2.7) 


39 (2.8) 


16(2.2) 


8(1.8) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. __ 
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Sample Geography Question 

Grade 12 — Map: Locate Physical Features 

Although the following question is similar to the preceding one, the features that the 
students were asked to locate were substantially less well-known than the continents 
fourth graders were asked to locate. This reflects the fact that twelfth graders are expected 
to have a greater body of factual geographical knowledge. 



For Question 5 , write your answer on the map below. 




5. Write the number of each of the following physical features 
in the correct location on the map on page 6. 

1 Pyrenees Mountains 

2 The Japanese Archipelago 

3 Mediterranean Sea 

4 Persian Gulf 



n 

i 



1 
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The scoring criteria paralleled those for the previous question. A “Complete” 
response correctly identified the locations of all four physical features on the map: the 
Pyrenees Mountains, the Japanese Archipelago, the Mediterranean Sea, and the Persian 
Gulf. An “Essential” response correctly labeled three of these four features. A “Partial” 
response correctly labeled one or two of the four features, and a response that did not 
correctly identify any of the features was scored as “Inappropriate.” Although students 
were asked to use the associated number to label the map, they were given full credit for 
writing the names of the features in the correct locations. Examples of student responses 
are presented below. 

Sample “Complete” Response 
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Sample “Essential” Response 




Sample “Partial” Response 
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Sample “Inappropriate” Response 




Information on how students performed on this question is presented in 
Table 3.7. Only a small percentage of twelfth-grade students (9 percent) were able to 
correctly locate all four physical features on the map. And, although most of the students 
attempted to answer the question, over a third of twelfth graders were not able to correctly 
locate even one of the four features. Although many students may have heard of these 
features, it is evident from their responses that many did not know their exact locations on 
a map. 



Table 3.7 


Score Percentages lor Map: 


THE N 
REPORT 
CARD 


lATION’S 

raep 


Locate Physical Features 




-i: 





Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 12 


9 (0.9) 


14(1.1) 


43(1.6) 


27(1.2) 


7 (0.6) 


Male 


11 (1-3) 


18(2.0) 


46 (2.2) 


20(1.6) 


6(1.1) 


Female 


8(1.1) 


11 (l.D 


41 (1.8) 


34(1.6) 


7 (0.9) 


White 


11 (1-2) 


16(1.2) 


45(1.9) 


24(1.4) 


5(0.7) 


Black 


4(1.2) 


5(1.5) 


33 (3.8) 


44 (3.2) 


15(2.5) 


Hispanic 


6(1.8) 


11 (2.5) 


44(3.8) 


28 (3.3) 


13(2.4) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Obtaining Information from Maps 

The geography assessment included numerous questions in which students were asked to 
draw upon information provided on a map to respond correctly to a question. In most of 
these questions, students were required to have some level of map literacy; for instance, 
they had to know how to use map conventions, such as keys, compass directions, scales, 
or lines of latitude and longitude. In order to answer many of these questions correctly, 
students also needed to provide other factual geographic knowledge about locations of 
places and natural features or some conceptual knowledge about geographical 
phenomena. 

Sample Geography Question 

Grade 4 — Map: Los Angeles to Salt Lake City 

All of the information students needed to answer the following question correctly is 
provided on the map. 



Questions 12-13 are based on the highway map below. 



13. To drive from Los Angeles to Salt Lake City in the most direct 
way, one would travel 

(a) southeast 

CD southwest 

9 northeast 

CD northwest 




MAJOR HIGHWAYS 



Highway route, 
mamty expressway 



Table by NYSTROM Division of Herff Jones, Inc. 
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In order to select the correct answer, students needed to understand compass 
directions and be able to read the highway map. The cities of Los Angeles and Salt Lake 
City are clearly noted on the map, the highway between those two cities is shown, and 
compass directions are also included on the map. However, what makes this question 
more difficult than a simple reading of information on the map is that the direction of 
travel is not directly east, west, north, or south, but in between two perpendicular 
directions. Student performance is presented in Table 3.8. The majority of fourth graders 
appeared to understand compass directions and were able to read the highway map. 



Table 3.8 


Percentage Correct for Map: 


THE h 
REPORT 
CARD 


lATiON’S 

reae p 


Los Angeles to Salt Lake City 




-vi; 



Grade 4 


61 (1.3) 


Male 


65(1.7) 


Female 


57(2.1) 


White 


66(1.7) 


Black 


42 (3.0) 


Hispanic 


55 (3.9) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 Geography Assessment. 
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Sample Geography Question 

Grade 4 — World Map: Width of Africa 

The following question tested students’ map literacy as well as their factual knowledge of 
a geographical location. 



THE WORLD 




2. About how wide is Africa at its widest point from west to east? 

CD 650 miles 
CD 1,500 miles 
® 3,500 miles 
CD 7,000 miles 



BEST COPY AVAILABLE 
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In order to answer the question correctly, students needed to know: 1) which 
land mass on the map was Africa, 2) the west-to-east direction, because a compass was 
not provided on the map, and 3) how to use the scale to measure for the widest distance. 
Student performance is presented in Table 3.9. Thirty-eight percent of students answered 
this question correctly. Although the question is relatively straightforward, it required 
students to bring together factual knowledge and knowledge of map conventions. 



fable 3.9 


Percentage Correct for 
World Map: Width of Africa 


THE N 
REPORT 
CARD 


lATlON’S 

raep 

-vj 




Grade 4 


38 (1.2) 






Male 


44 (2.0) 






Female 


32(1.4) 






White 


42(1.5) 






Black 


24 (3.3) 






Hispanic 


34 (3.7) 







Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 Geography Assessment. 
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Sample Geography Question 

Grades 8 and 12 — Earthquake Map: Where Most Occurred 
The following is a relatively straightforward question in which students were asked to use 
the information provided on the map to respond. However, students also needed to use 
the key to interpret what was presented on the map. 



Questions 3-4 are based on the map below showing the distribution of earthquake 
epicenters around the world between 1961 and 1967. 

0 ° 60 ° 120 ° 180 ° 120 ° 60 ° 0 ° 



3. Between 1961 and 1967, the area that had the most earthquakes was the 

CD Mediterranean basin 
CD mid- Atlantic Ocean 
CD Caribbean Sea 
• Pacific Ocean rim 
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To answer this question correctly, students needed to use the key to understand 
that the density of dots indicated the number of earthquake epicenters distributed across 
the map. As shown in Table 3.10, very high percentages of both eighth and twelfth 
graders answered this question correctly. As expected, twelfth graders did somewhat 
better than eighth graders. Although the question was testing students’ map literacy, the 
fact that the Pacific Ocean was clearly labeled and was prominently centered on the map 
may have helped some students to answer this question correctly. Furthermore, students 
may have had prior knowledge about the prevalence of earthquake activity in this area 
and perhaps only needed the map to confirm what they already knew to be the correct 
answer. 



Table 3.10 


Percentage Correct for 
Earthquake Map: 
Where Most Occurred 


THE A 
REPORT 
CARO 


IAT10N’ 

rcaefj 


s 

i 

5 : 


Grade 8 Grade 1 2 



tal 


84 (0.7) 


91 (0.9) 


Male 


83(1.2) 


92(1.2) 


Female 


85(1.1) 


91 (1.2) 


White 


87(1.0) 


93(1.0) 


Black 


75 (2.7) 


84 (2.1) 


Hispanic 


79 (2.5) 


87 (3.4) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



Interpreting and Explaining 

In addition to being presented with questions in which the primary skill demonstrated was 
reading information from maps, students were also presented with questions that required 
them to combine the information given on the maps with outside knowledge in order to 
explain such matters as: 1) geographical events, 2) how geographical features can impact 
people’s lives, or 3) relationships between geography and historical occurrences. For 
example, students were presented with a map of ocean currents and asked to explain how 
specific currents affected the economy of a given country. Or similarly, students were 
presented with a map of population migration patterns and asked to explain these patterns 
based on outside knowledge of history and geography. The outside knowledge students 
were asked to bring to these questions varied widely. Some of the knowledge was 
directly related to geographical phenomena while other knowledge was of a social, 
political, or economic nature. 
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Sample Geography Question 

Grade 4 — Street Map: Bridge Closing 

In this and similar questions, students were asked to judge the impact a change in the 
geographic surroundings would have on people’s lives. 



MILLTOWN 




4. If Central Bridge is closed for repairs, what will happen in Milltown? 

(D Traffic will move faster. 

♦ Upper Bridge will have more traffic. 

(cD Central Avenue will have more traffic. 

CD The shopping center will close down. 



8 
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To answer this question correctly, students needed to be able to read the street 
map and to have an understanding of traffic patterns. That is, students needed to see that 
people living in the area from Front Street to Third Street would need to use either the 
Central Bridge or the Upper Bridge to get to the school, the shopping center, or the steel 
mill. Therefore, the most likely outcome of a bridge closing on Central Bridge would be 
that traffic would be diverted to the remaining river crossing, and Upper Bridge would 
have more traffic. Student performance is presented in Table 3.1 1. Over half of the 
students were able to identify the correct answer. The most commonly selected incorrect 
response was option C, “Central Avenue will have more traffic,” with about one-fourth of 
students selecting it. 



Table 3.11 



Percentage Correct for 


THE h 
REPORT 
CARO 


iATION’S 

raep 


Street Map: Bridge Closing 







Grade 4 


57(1.3) 


Male 


62(1.8) 


Female 


53 (2.0) 


White 


65(1.7) 


Black 


32 (2.8) 


Hispanic 


47 (4.1) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 Geography Assessment. 
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Sample Geography Question 

Grade 8 — Map: Locate Most Populous City 

The following question is representative of many others that asked students to evaluate 
the impact of geographical features on people’s lives. In these types of questions, students 
needed to understand map conventions and have knowledge of how geography affects 
people’s lives. 



GRAND ISLAND 




10. The island shown on the map above has four cities — A, B, C, and D. 
Based only on the geographical features shown on the map, which city 
probably has the largest population? 



• 


City A 


CD 


City B 


CD 


City C 


CD 


City D 



S3 



74 



Using the Tools and Resources of Geography and U.S. History 



To answer this question correctly, students needed to be able to use the key on 
the map to identify the physical surroundings of the four cities, and then apply their 
knowledge of how natural features such as mountains and rivers influence people’s 
choices about where to live and build cities. Student performance is presented in 
Table 3.12. 



Table 3.12 



Percentage Correct for Map: 


THE h 
REPORT 
CARO 


lATION’S 

rcaep 


Locate Most Populous City 







Grade 8 

Male 

Female 

White 

Black 

Hispanic 



58(1.4) 

62 (2.4) 
54(1.9) 
65(1.8) 
33 (2.7) 
46 (4.0) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational 
Progress (NAEP), 1 994 Geography Assessment. 



Nearly 60 percent of students correctly chose City A. It appears that a majority 
of eighth-grade students understood the impact of rivers and mountains on growth of 
cities. There was still a substantial proportion of students, however, who did not 
understand the question, were not able to read map features, or lacked knowledge of the 
geographic factors that favor urban development. 
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Sample Geography Question 
Grade 12 — Map: Religious Centers 

The following question is representative of a group of questions in which students were 
asked to consider places identified on a map and decide what common factor 
characterized all of the places in the set. In order to answer this question correctly, 
students needed to have outside knowledge about religious centers and be able to read 
the map. 




2. The four locations indicated on the map above are 

CD capitals of highly industrialized nations 
CD the world’s four most densely populated cities 
CD areas of highest elevation 
• religious centers 



C r- 

uO 



76 



Using the Tools and Resources of Geography ancl U> S. Hisfon' 




To correctly answer the question shown here, students needed to know that the 
four named cities are each of particular importance to a specific world religion. A similar 
question could have been asked without using a map as a stimulus (for example, “How 
are Rome, Jerusalem, Mecca, and Benares similar?”), but students with partial knowledge 
may have been aided by their ability to use the additional clues that the map provided 
about the cities’ locations. In particular, many U.S. students may have been less familiar 
with the religious associations of Benares than with those of the other three cities. 
However, if they knew that India was a major Hindu country and saw that Benares was 
located in India, they could extrapolate from their knowledge of the religious significance 
of Rome, Jerusalem, and Mecca to correctly select option D. Further, by physically 
locating the cities, the map could help students draw upon their general geographic 
knowledge to eliminate other options such as C, “areas of highest elevation.” 

Student performance is presented in Table 3.13. Over three-fourths of twelfth- 
grade students were able to answer this question correctly. It appears that students had 
some knowledge about these cities and were able to use the knowledge they had to select 
the best answer to the question. 
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Sample Geography Question 
Grade 8 — Map: Language Patterns 

In the following question, the map used shadings to indicate different areas of dominant 
language families. Students were asked to use the information provided on the map and 
their outside knowledge about language families and history — specifically, those of 
indigenous peoples, conquests, and patterns of migration and settlement — to answer the 
question. 



LANGUAGE FAMILIES 




15 . What historical trends explain the language patterns shown in the map above? 
Be as specific as possible in your answer. 



This is an extended constructed-response question. A “Complete” response 
provided a specific historical explanation for the pattern observed, including a discussion 
of the survival of non-European languages in some regions. Credited responses included 
the fact that Indo-European languages in North, Central, and South America were brought 
by the English, French, and Portuguese colonizers after the 15 th century and that 
Amerindian and Eskimo- Aleut languages, spoken in the Americas prior to the 15 th 
century, managed to survive in remote, isolated areas. An “Essential” response provided 
an explanation of either the colonizing influence or of the languages of the people living 
in the Americas prior to colonization. A “Partial” response provided a general, rather than 
a specific, explanation, such as the fact that languages spoken in a country are generally 
of the country that first owned them. An “Inappropriate” response provided no 
appropriate explanation of the language patterns shown. Examples of student responses 
are presented below. 
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Sample “Complete” Response 



15. What historical trends explain the language patterns shown in the map above? 
Be as specific as possible in your answer. 



tQ»l CeUtiAdo alt, i/tl ritlktA. l*£O.U6i. 

an* pXkdAj tA c&dd&sL M&d. i 

it> *)Mxr ynu. /at - M* j0a k 

ueL ta u aaotai^ Jj^fuaL fiipk caoa tJ. Axxid- £&D£*m£ &24dt — 
gagiLk Xa. Ha£tf& of*-* <WuMr hmisL 

It mu s)irM it a 0 *L. d/md* Jrx O/fUkhtc/K, <fih £. — — 

^BoAirt- au . J^mve. -jAt <£&Qk — aJ\ ! AA - 1 S k — 



fo&a^L otuL l* 



/)ntW , 



Sample “Essential” Response 

15. What historical trends explain the language patterns shown in the map above? 
Be as specific as possible in your answer. 

jazuohl&l — claims*, — hy ^ c 

n< by fe/iefan romnrlA^g — 0»< r ~1 i*t *> 

pe&pls. b*j On lb b — l1_£ — OUJs& l 



Sample “Partial” Response 



15. What historical trends explain the language patterns shown in the map above? 
Be as specific as possible in your answer. 

Jl Jo fy Uy** OM itttjd If"* 1 ^ 

•J* Ail^ Aii»tLhj>c£j£tJ* 
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Sample “Inappropriate” Response 



15. What historical trends explain the language patterns shown in the map above? 

Be as specific as possible in your answer. 

- uJU l> Vi£ bJ«t W 

t Vs Pfd^ie \1>6T 

_|q n^cng e. 

Information about student performance is presented in Table 3.14. Very few 
eighth graders were able to provide even a partially correct response for this question. 
Over half of the students provided responses that were scored as “Inappropriate,” and 
another 28 percent chose to omit this item. The majority of students who received credit 
for their responses were able to provide only partial answers. There are a couple of 
possible reasons why this was a challenging question. First, the topic of language families 
may not be a common one in students’ curricula. Second, the question provided no 
guidance to students as to the type of historical explanation to consider in explaining the 
patterns presented on the map. This open-endedness, in light of the unfamiliarity of the 
topic under discussion, may have made the question particularly difficult for students. 



Table 3.14 


Score Percentages for Map: 


THE b 
REPORT 


iATlON’S 

raqpl 


Language Patterns 


LAnU 


=£: 





Complete 


Essential 


Partial 


Inappropriate Omit 


Grade 8 


2 (0.3) 


6 (0.6) 


12(1.1) 


52(1.6) 


28(1.0) 


Male 


2 (0.4) 


5 (0.9) 


12 0.4) 


48(1.8) 


33(1.6) 


Female 


2 (0.5) 


7 (0.9) 


11 (1.4) 


56 (2.2) 


23(1.6) 


White 


2 (0.4) 


8 (0.8) 


14(1.3) 


51 (1.9) 


25(1.3) 


Black 


1 (0.4) 


3(1.5) 


4(1.2| 


55 (3.2) 


38 (2.5) 


Hispanic 


1 (0.5) 


3 (0.9) 


8(1.8) 


51 (3.7) 


38 (2.9) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Sample Geography Question 

Grade 12 — Map: Letter to Support Shopping Site 

The following is one of a number of questions that asked students to consider the impacts 
of geography on people’s lives. This question is unique in that it asks the students 
themselves to make a choice and to argue convincingly for that choice in a letter to the 
mayor of the city. 



MADDIEVILLE 




9. Maddieville is building a new shopping center. There is a disagreement 
in the city council over whether to build the shopping center at site 
A or at site B on the map on page 10. 

As a resident of the city who would like to shop at the new shopping 
center, write a letter to the mayor in support of either site A or site B. 
Give three reasons why the site you support is better than the other 
site. 



This is an extended constructed-response question. A response was scored as 
“Complete” if a student selected a site and provided three appropriate reasons to support 
this selection. Possible reasons for site A included the fact that it was located near public 
transportation and near the central business district; it would keep the downtown area 
vital; it was a site that was familiar to potential customers; and the central location 
maximized accessibility. Possible reasons for site B were that there was more room to 
build in that area; land was probably less expensive; parking would be easier for 
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customers; it was near an interstate that could possibly bring in more customers; it would 
diffuse congestion in the city; and it could lead to other land development around the mall 
— for example, a park for recreational purposes. An “Essential” response chose a site and 
provided two credible reasons for the choice. A “Partial” response chose a site and 
supported the site with one reason. An “Inappropriate” response may or may not have 
selected a site, but it did not offer any plausible support for the location. Although 
students were asked to write a letter they were given credit regardless of whether they 
responded with a letter format. Examples of student letters to the mayor are presented 
below. 



Sample “Complete” Response 



kvcWiau. \\r ix \o -&A ^ n 









A . iW <dM> 






\ Ar&oV U 



J ^ 



or\ Sfojfiw oA, T.rAjtifo\*^ PfcfocV j 

SfpgiJb aA jt\jL fSj, yVsd^ 






SVNa^JA' 




o 
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Sample “Essential” Response 






“fV 



J qjw\ isaa&lL 



xlqKjv»a^ o Jivi dn. 

/nOL'v' /Xa^UT* < fL/f^UC Ut*» -ft . Jfdx. 

~E5> U>PU^ o. VtirfWV^Q AiA ctt- -fe- WaLI ^ 

CLtAlU , ott UJOixU Itt-iflA ^ 






,tW 



ityfiyOfAhti 

)\tr AAWvSL. Jr Ik. jiV^vmc 

J4 iiaouS^ Prtl/AlL A. (^VfO \^ /naxv OkumSsM# 

-ter tidhJU v /iy/^~4 W. /HIaM <A~ Ajdi- ^ ura<t^ k 

t/^a/nd, -Hhc. rrbiy /i^iA -^U. 2£jzfi&Al^ 

;dV\A/n\a Vg<^> ) 



&• o/xftrtumii 
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Sample “Partial” Response 









U.»*g/yAJh N^k?U y^v^X^U^v^ - 

3^ 4*. JntMJ- vx»J- woJa 

& ft Jr ^UL/ Xtv jfcki^ V^vludLAibc fr|- 

t/A-XM/4vJL\jL^ ^OMuis. JJsiAJO OUvi _ 

^AA/g^Y^ ^AX 4 ryyv 4 yir>^ J^CKa^m^/v. 

OA^A. > JUfec ft WaU. 



U/r^ 



U&y . ^)-Wi 



»Liva*\a y- AAA AXe^ 'T^AupW A- 
v AA wO«Ax ^jJl?f-- 

urvs^oJl^ fi 1 1 il «J4s-*4 



g\^r 

\.^vwg\>i^ ^|m AJlC^J aA/ 

Jckp V^olky, , 



/ T-Ka^J^ — -yffrs. ■AwhA' # K-(3JU 
utygju^AX.. 



Us. AzLjl^ 



^UAU^b M^t\* 
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Sample “Inappropriate” Response 
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Information on student performance is presented in Table 3.15. Fifteen percent 
of twelfth-grade students were able to provide three appropriate reasons for the site they 
selected. Another 40 percent were able to provide at least two appropriate reasons for 
their selection. Only 10 percent of students were not able to provide any appropriate 
reason in support of their site, while 7 percent omitted the question altogether. Although 
twelfth-grade male students outperformed females on map questions in general and also 
on the geography assessment as a whole, on this question a higher percentage of female 
students provided “Complete” or “Essential” responses than did their male counterparts. 




Creating Maps 

The geography assessment included several questions that asked students to demonstrate 
their ability to draw maps from textual descriptions. In order to correctly create maps, 
students needed to have knowledge about maps as well as specific geographic knowledge. 
Students were asked to create a variety of maps. For example, in one of the questions 
in the eighth-grade assessment, students were asked to create a weather map from a 
description of weather conditions in different areas of South America. In another 
question, students were provided with a description of a town, information about the 
main street, and the locations of features such as a park and a school. They were then 
asked to draw a map of the town using an appropriate scale and providing a key to the 
symbols used. 
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Sample Geography Question 
Grade 4 — Draw Island Map 

In the following question, students were asked to create a map from a written description. 
The description was straightforward. To answer the question correctly, however, students 
needed to know that an island is land wholly surrounded by water, and they needed to be 
able to use symbols and compass directions to accurately locate specific features on the 
island. 



12. 1) In the box below, draw a map of an island. 

2) On the island, put in the following details: 

- Mountains along the west coast 

- A lake in the north 

- Houses along the east coast 

- Forests in the south 

Be sure to use the symbols shown in the key below. 
Use your colored pencils to help you draw the map. 






































































































































































Key: 

Mountains Forests 

Cf) Lake n 

666 Hous<a w ~“p E 
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This was an extended constructed-response question. A response scored as 
“Complete” include all the elements described. These elements included the following: 

1) the perimeter of the island enclosed by a continuous line; 2) mountains placed to the 
west (left) of an imaginary north-to-south line and as close to the western perimeter as 
possible; 3) a lake constructed to the north of (above) an imaginary east-to-west line; 

4) houses drawn to the east (right) of the north-south line, as close to the eastern 
perimeter as possible; and 5) a forest drawn anywhere within the perimeter, south of 
(below) the imaginary east-to-west line. 

An “Essential” response had three or four of the five elements mentioned above. 
Also, a response was scored as “Essential” if it correctly included the island and all of the 
features, but some of the features were not placed correctly. A “Partial” response had one 
or two of the elements mentioned above. An “Inappropriate” response did not include any 
of the elements mentioned. Examples of students’ responses are presented below. 
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Sample “Complete” Response 







7 






















A 


y\ I 










i 


// 


V 




V' 


jWHITwPB 

aU'iflM 
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Mountains Forests 

CO Uke ? 

666 Houses »•- j- -*« 



Sample “Essential” Response 
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Sample “Partial” Response 






90 



Using the Tools and Resources of Geography and U.S. History 



Student performance is presented in Table 3.16. Forty-four percent of fourth- 
grade students were able to draw an island and correctly place the mountains, lake, 
houses, and forest. Only a very small percentage omitted the question or drew a map that 
had no appropriately drawn elements. It thus appears that many fourth-grade students 
were able to read explicit descriptions of geographic features and translate them to maps. 



fable 3.16 


Score Percentages for Draw 
Island Map 


THE H 
REPORT 
CARD 


lATlON’S 

rcaep 




Complete Essential Partial Inappropriate Omit 



Grade 4 


44(1.5) 


26(1.6) 


18(1.1) 


6(0.7) 


6 (0.8) 


Male 


44 (2.0) 


25 (2.3) 


17(1.6) 


6 (0.9) 


8(1.0) 


Female 


43 (2.1) 


27(1.8) 


19(1.7) 


6 (0.8) 


4(0.7) 


White 


51 (2.1) 


25 (2.0) 


15(1.4) 


4(0.7) 


4(0.7) 


Black 


17(2.1) 


28 (2.8) 


27(3.1) 


14(2.5) 


15(3.0) 


Hispanic 


33 (3.9) 


26 (4.0) 


24 (2.9) 


9 (2.3) 


8 (2.2) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 



Sample Geography Question 
Grades 8 and 12 — Draw Isthmus Map 

The following question represents a similar, but more demanding, map drawing exercise 
that was given to both eighth and twelfth graders. Here, students had to extract the proper 
map-making directions from the prose account of a traveler rather than being explicitly 
told what elements to include and where to place them as in the previous example. In this 
question, students also were asked to include a scale to indicate distance. Students were 
not asked to provide a key, although a key was a requirement in other, similar questions. 
This question assessed students’ ability to create simple maps, their understanding of 
compass directions, and their knowledge of the definition of an isthmus. 
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11. After we anchored our ships in the ocean and went ashore to explore, we 
marched west. The forest was so thick we could only travel three miles in 
the first two days. Then we came to the mountains and climbed to the top. 

A rushing river flowed west out of the mountains. We continued to march 
two miles west and came down out of the mountains. Two miles further we 
came to the coast. It was obvious that the area we were exploring was an 
isthmus. 

In the box below, draw a map of the region described above. Be sure to include 
all of the geographical elements mentioned in the description. Include a scale to 
indicate distances. 



N 

W E 

j; 



For this extended constructed-response question, “Complete” answers included a 
map that had four correctly placed features and showed some indication that the map was 
of an isthmus. Features that could be included were the east coast, the forest, the 
mountains, the river flowing west, and the west coast. The placement of the different 
features had to be in the right direction. Although the question asked for a scale, the use 
of a scale was not necessary to get a “Complete” score. Also, students could receive full 
credit if they chose to create a relief map. 

An “Essential” response included a map with three features correctly placed. If 
the response showed an island or a peninsula rather than an isthmus, but included three 
correctly placed features, a response was still scored as “Essential.” A “Partial” response 
included a map with at least two features correctly placed. An “Inappropriate” response 
did not include a map or showed none of the features correctly placed. Sample student 
responses are presented below. 
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Sample “Complete” Response 




Sample “Essential” Response 
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Sample “Partial” Response 




Sample “Inappropriate” Response 
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Student performance is presented in Table 3. 17 by grade level. Although 15 
percent of eighth graders were able to provide a response that was rated as “Complete,” 
20 percent of the students provided maps that had no correctly drawn features. It appears 
that 65 percent of eighth-grade students were at least able to understand and correctly 
draw some of the information provided in the narrative. A larger percentage of twelfth- 
grade students, 25 percent, were able to provide “Complete” responses, and only a tenth 
of the twelfth-grade students provided responses that had no correctly drawn features. 



Table 3.17 Score Percentages for Draw Isthmus Map 



THE NATION’S 
REPORT 
CARD 



m 





Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 8 


15(1.2) 


27(1.2) 


23 (1.1) 


20(1.2) 


16(1.0) 


Male 


16(1.4) 


28(1.6) 


23(1.7) 


18(1.7) 


16(1.5) 


Female 


14(1.6) 


26(1.8) 


23(1.5) 


22(1.9) 


16(1.5) 


White 


19(1.7) 


31 (1.5) 


23(1.7) 


17(1.6) 


11 (1.2) 


Black 


3 (0.7) 


1 1 (2.2) 


22 (2.9) 


29 (2.6) 


35 (3.1) 


Hispanic 


1 1 (2.5) 


18(2.9) 


23 (3.0) 


26 (2.6) 


23 (2.6) 


Grade 12 


25 (1.2) 


34(1.4) 


17(1.0) 


10(0.9) 


13(1.0) 


Male 


28(1.9) 


34(1.7) 


16(1.3) 


9(1.4) 


12(1.2) 


Female 


23(1.7) 


35 (2.0) 


18(1.6) 


11 (1.2) 


14(1.2) 


White 


29(1.6) 


38(1.8) 


16(1.2) 


8(1.0) 


9(1.1) 


Black 


7(1.3) 


21 (2.4) 


19(2.6) 


19(2.9) 


34 (3.8) 


Hispanic 


18(3.5) 


26 (3.3) 


20 (3.5) 


14(2.4) 


22 (2.4) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 


1 994 Geography Assessment. 











Summary 

In the 1994 geography assessment, many students in all grades were able to locate places 
and physical features on maps. The difficulty of questions depended in large part on the 
level of outside knowledge students had to bring to bear on the question. That is, fewer 
students were able to answer questions correctly when they required factual knowledge of 
generally lesser known places or natural features. 

Most students at all grade levels appeared to have some degree of map literacy. 
For example, the majority of fourth graders were able to locate the equator, the north 
pole, and the south pole; and almost half of eighth graders were able to identify parallels 
of latitude. However, it appears that there were still many students with low levels of map 
literacy. For example, many students were able to find the directions north, south, east, or 
west, but had greater difficulty with directions such as southeast or northeast. Similarly, 
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many students appeared to understand the concept of a scale, but fewer were adept at 
using one. Nevertheless, many students understood how to create and use a variety of 
map conventions. A substantial proportion of students were able to draw a partially 
correct map from a textual description, but the majority were not able to draw, or were 
not careful about drawing, the completely correct rendition. 

Based on the questions that required students to apply map-based knowledge to 
explicate human events, it appears that eighth- and twelfth-grade students have been 
exposed to ideas about how natural geographic features and changes in people’s 
environments impact people’s lives and the decisions they make. Many students, 
however, found it difficult to clearly explain their understanding of these concepts 
in writing. 
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Chapter 4 



Working with Charts, Graphs, 

and Tables 



Charts, graphs, and tables are tools used to communicate information. When constructed 
accurately, these tools allow a great deal of data to be summarized succinctly and 
facilitate the understanding of trends and relationships. The ability to comprehend charts, 
graphs, and tables has become essential in today’s world. These tools are not simply 
auxiliary aids; they are used to argue positions and to influence decision-making. For 
example, students regularly encounter charts, graphs, and tables in daily newspapers, on 
television programs, in school textbooks, and in many other resource materials. To be 
wise consumers of information presented in the media, and to be effective and ethical 
users of such tools, today’s children must be able to interpret, use, and construct charts, 
graphs, and tables. 

Both the 1994 NAEP geography and U.S. history frameworks emphasized the 
importance of using charts, graphs, and tables in assessment questions. Assessment 
developers were called on to craft tasks that measured students’ abilities to read and 
interpret simple quantitative data and to place such data in appropriate contexts. Because 
achievement in history and geography also includes the ability to communicate one’s 
understanding to others, the frameworks further suggested that students be asked to create 
charts, graphs, and timelines as part of the NAEP assessments. 
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Graphic Questions in the 

1994 Geography and U.S. History Assessments 

Both quantitative and qualitative data were presented in assessment questions involving 
graphs, tables and charts; however, quantitative data predominated. The emphasis in the 
assessments was on using such data in conjunction with other factual knowledge to 
explain matters of geographical and historical significance. More specifically, students 
were required to do the following: 

• Extract information presented in charts, graphs, and tables. 

• Identify and interpret trends and patterns represented by the data. 

• Explain data in the charts, graphs, and tables by using outside knowledge 
of geography and history. 

• Construct graphic representations of data presented. 

The number of questions involving graphs, tables, or charts in the 1994 NAEP 
geography and U.S. history questions are shown in Table 4. 1 below. 



Table 4.1 


Number of Questions Involving 
Graphs, Charts, and Tables on the 
1994 NAEP Assessments in 
Geography and U.S. History 


THE NATION’S 
REPORT ram 
CARD ™ 




Geography 


History 




Grade 4 


5 


3 




Grade 8 


8 


16 




Grade 12 


10 


18 




Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 994 U.S. History Assessment. 
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Examples of 1994 NAEP Geography and U.S. History 
Questions Involving Graphs, Tables, or Charts 

Students’ performance on individual questions in this category suggests that the difficulty 
of a particular question was related to three main factors: 1) the students’ ability to read 
and interpret the data in the charts, graphs, or tables; 2) the type and amount of outside 
content knowledge that students were expected to bring to bear in answering the question; 
and 3) what students were asked to do with the information in the charts, graphs, and 
tables — that is, the level of inference at which students were asked to perform. Other 
things being equal, students were more likely to answer correctly a question that asked 
them to pick out individual bits of information from a chart, graph, or table than one that 
asked them to identify a pattern or trend. Furthermore, students were more likely to 
answer correctly a question that asked them to identify a trend than they were a question 
that asked them to provide appropriate geographical or historical explanations for that 
trend. 

In the 1994 geography and U.S. history assessments, students were presented 
with a variety of graphic stimuli, including bar charts, line graphs, pie charts, timelines, 
text tables, and quantitative data tables. Examples of actual assessment questions are 
shown on the following pages. These examples illustrate how questions can ask students 
to demonstrate content knowledge as well as reading and interpretive skills. The 
questions are presented based on the dominant skill required to answer the question 
correctly — data extraction, trend identification, explanation of the information, or 
construction of graphic presentations. Also considered is the complexity of the outside 
content knowledge students were required to bring to the task. 

Extracting Information 

To correctly respond to the questions highlighted in this section, the dominant skill 
students needed was the ability to read the data presented in the stimuli. For some 
questions, all of the information required to answer correctly was contained within the 
stimulus. In other questions students were required to draw upon their own factual 
geographical or historical knowledge. 
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Sample U.S. History Question 

Grade 4 — Timeline: Mayflower and Thanksgiving 

The following question was about the Pilgrims’ journey to and settlement in America. 

Students were asked to use the timeline to answer the question. 



First Year in Plymouth, from Fall to Fall 





Mayflower Arrives 



1620 

Sept. 


Oct. 


Nov. 


Dec. 


1621 

Jan. 


Feb. 


March 


April 


May 


June 


July 


Aug. 


Sept. 


Oct. 



Leaves 

England 




With Crops 




i IldAIIVOgl V 



3. What can you tell from the time line? 

(a) The Mayflower took more than one year to sail to Plymouth. 

CD The Mayflower arrived in Plymouth in 1621. 

• The first Thanksgiving was not celebrated in the same month 
as it is today. 

(D The Pilgrims in Plymouth ate mostly fish. 
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In order to select the correct option, students had to be able to read and 
understand the timeline, which indicates that the First Thanksgiving was celebrated in 
October rather than November, as it is today. In order to eliminate the first two options, 
they needed to understand from the time line that the Mayflower was at sea for about four 
months and that it arrived in America before the beginning of 1621. Information on 
student performance for this question is presented in Table 4.2. 





Percentage Correct for 


THE NATION’S 


Table 4.2 


Timeline: Mayflower 


REPORT 

CARD 


raq i 

3 


i 




and Thanksgiving 




—4 





Grade 4 

Male 

Female 

White 

Black 

Hispanic 



32(1.4) 
29(1.8) 
36(1.9) 
32(1.7) 
35 (3.0) 
27 (3.2) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 U.S. History Assessment. 



A third of the fourth graders were able to answer this question accurately, 
showing that they were able to read the timeline, understand the negative phrasing of the 
correct response option, and relate it to their outside knowledge that Thanksgiving today 
is not celebrated in October. However, it appears that many fourth graders were not able 
to correctly extract the appropriate information from this timeline. Over a third of the 
students, 38 percent, selected option B, which erroneously states that the Mayflower 
arrived in 1621. Unlike their overall performance on the U.S. history assessment, female 
students performed better than male students, and there was no significant difference 
between racial/ethnic subgroups on this question. 
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Sample Geography Question 

Grade 8 — Bar Graph: Where Are the Largest Oil Reserves ? 

In the following question, students were expected to read a bar graph and draw on outside 
factual knowledge to respond. 




1. According to the graph above, the largest oil reserves in 1986 were in the 
< 3 > Gulf of Mexico 
(D Caribbean region 
^ Persian Gulf region 
(d) Gulf of Guinea 
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To answer this question correctly, students needed to read the graph to determine 
which country had the largest oil reserves in 1986, and to apply outside knowledge to 
properly place the countries in relation to the world regions shown in the answer options. 
Student performance is presented in Table 4.3. 





Percentage Correct for 


THE NATION’S 


Table 4.3 


Bar Graph: Where Are 


REPuH 1 
CARD 






the Largest Oil Reserves? 







Grade 8 


85(1.0) 


Male 


87(1.3) 


Female 


82(1.4) 


White 


89(1.3) 


Black 


76(1.7) 


Hispanic 


72 (2.7) 


Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 


Educational Progress (NAEP), 1 994 Geography Assessment. 



This question was quite easy for students in grade 8. A large majority of eighth 
graders appeared able to read a bar graph with categories, in this case countries along the 
x-axis and percentages along the y-axis. These students also appeared to know that Saudi 
Arabia is in the Persian Gulf region. The relatively recent focus on that region because of 
the Persian Gulf War may have helped students recall this geographic fact. 

The difficulty of the question was also influenced by the fact that 1986 oil 
reserves in Saudi Arabia so far outstripped those of any other country. Had the 
distinctions among countries been less extreme, the students might have had to add 
together the reserves of the three Gulf region countries in order to verify their answer. 
The latter task would have made greater demands on their graph reading and quantitative 
skills. 
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Sample Geography Question 

Grade 12 — Quantitative Table: Population of State X 

The following question was part of a set of questions that presented students with two 

tables of information regarding the population of a hypothetical state. Here we have 

another example of a question which asks students to extract information from a tabular 

display. 



Questions 7-8 are based on the following tables. 



POPULATION OF STATE X 



Year 


Total Population 


Urban 


Rural 


1850 


92,600 


7% 


93% 


1860 


378,000 


21% 


79% 


1870 


560,200 


37% 


63% 


1880 


864,700 


42% 


58% 


1890 


1,213,400 


49% 


51% 


1990 


1,485,100 


52% 


48% 


1910 


2,377,500 


62% 


38% 


1920 


3,426,900 


68% 


32% 


1930 


5,677,300 


73% 


27% 


1940 


6,907,400 


71% 


29% 


1950 


10,586,200 


81% 


19% 


1960 


15,717,200 


86% 


14% 


1970 


19,953,100 


91% 


9% 


1980 


23,668,600 


91% 


9% 




PEOPLE WHO MOVED TO STATE X 






Years 


Number of People 
Moving into State 






1870- 1880 


129,600 






1880- 1890 


214,200 






1890- 1900 


172,700 






1900- 1910 


694,100 






1910- 1920 


804,100 






1920- 1930 


1,695,200 






1930- 1940 


974,600 






1940- 1950 


2,399,100 






1950 - 1960 


2,780,100 






1960- 1970 


1,528,000 






1970- 1980 


1,462,000 




During which ten-year period did the percentage of people living in 
urban areas increase the most? 



• 1860-1870 
CD 1890-1900 
(c) 1930-1940 

<D 1960-1970 113 
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For this question, all of the data needed to answer the question are included in 
the first table. The information in the second table is not relevant because it does not 
separately distinguish the urban population. Having identified the correct data source, 
students also needed to perform simple mathematics in order to correctly select option A. 
Specifically, students needed to: 1) recognize that the requisite data were included in 
columns one and three of the first table; 2) know that for each ten-year period being 
considered, they needed to subtract the urban percentage of the previous decade from the 
percentage of the later decade; 3) do the subtraction of whole numbers; and 4) compare 
differences and select the decade with the largest positive difference. Student 
performance for this question is presented in Table 4.4. 



Table 4.4 



Percentage Correct for 
Quantitative Table: 
Population of State X 



THE NATION’S 
REPORT fr apn 



CARD 



M: 



Grade 12 


65 (1.3) 


Male 


66 (2.0) 


Female 


65(1.8) 


White 


68(1.6) 


Black 


53 (3.0) 


Hispanic 


49 (3.0) 



Standard errors of tfie estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



Nearly two-thirds of the students correctly identified 1860-1870 as the decade 
during which the population of urban areas increased the most. This performance appears 
to indicate that a majority of twelfth graders were able to read quantitative information 
from a table and do simple mathematical operations. The pattern of incorrect answers 
may provide some insight into why the percentage correct was not higher. Option D was 
the most popular incorrect option, with 27 percent of all students selecting it. Thus, 
students might have understood the concept of increasing population but erroneously 
focused on the “Total Population” column rather than the “Urban” column. 
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Identifying Patterns and Trends 

Both the geography and U.S. history assessments from tables, charts, and graphs included 
questions that required students to go beyond simply extracting information to identify 
trends or patterns represented by the data and relate them to outside geographical or 
historical information or concepts. 

Sample Geography Question 

Grade 4 — Text Table: What Analysis of Export Chart Shows 

The following question was based on a text table of the major exports of three countries. 

This question asked students to read the table and draw on outside knowledge to respond. 



MAJOR EXPORTS OF THREE COUNTRIES 


Country A 
Oil 

Natural Gas 
Coconuts 


Country B 
Cars 

Televisions 

Cameras 


Country C 

Computers 

Airplanes 

Wheat 



6. The situation shown in the chart will probably lead to 
^ trade among all three countries 
(D trade only between countries A and B 
( 5 ) trade only between countries B and C 
CD a decision by each country to produce all nine goods listed 
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In order to answer the question correctly, students needed to read the table and 
determine that none of the countries had any major exports in common and that at least 
one of each country’s exports would likely be desired by the other two countries. They 
also had to understand the concepts of exporting and importing. Student performance 
information is presented in Table 4.5. 



Table 4.5 


Percentage Correct for Text Table: 


THE h 
REPORT 
CARD 


lATlON’S 

rcaep 


What Analysis of Export Chart Shows 







Grade 4 


37(1.6) 


Male 


39(2.1) 


Female 


34 (2.2) 


White 


41 (2.3) 


Black 


23(2.5) 


Hispanic 


26 (3.6) 


Standard errors of tfie estimated percentages appear 


in parentheses. 


SOURCE: National Center for Education Statistics, National Assessment of Educational 


Progress (NAEP), 1 994 Geography Assessment. 





A little over one-third of the students answered this question correctly. However, 
nearly as many students, 36 percent, incorrectly selected option D for their answer. It 
appears that many students were able to read this text table and determine that none of the 
countries export the same products; on the other hand, a substantial number apparently 
did not understand the trade implications of this information. 
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Sample U.S. History Question 

Grade 12 — Line Graph: Live Births in the United States 
The following question provided students with a line graph showing the rate of live births 
in the United States over time. Students were required to read the graph, identify trends in 
births over time, and recall information about dates of major historical events in the 
twentieth century in order to identify the correct option. 





LIVE BIRTHS PER 1,000 RESIDENTS IN THE UNITED STATES 




30 






25 


- 




§20 

pO 






! 15 






10 






5 


1 1 1 1 1 1 1 1 1 1 1 






1920 1925 1930 1935 1940 1945 1950 1955 1960 1965 1970 
Year 




12. The graph above supports which statement about the birthrate in the United 
States? 

(a) It declined steadily from 1920 to 1950. 

(D It increased rapidly during the Great Depression. 

^ It increased after the Second World War. 

(d) It tended to increase after each war. 



In order to answer this question correctly, students needed to read the line graph 
and have knowledge of the dates of the Great Depression, the year the Second World War 
ended, and the dates of other significant wars in which the U.S. was involved during the 
twentieth century. Only option A can be discarded based solely on information provided 
on the graph, because that option is couched in terms of the decades labeled on the 
horizontal axis. Student performance on this question is presented in Table 4.6. 
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Table 4.6 



Percentage Correct for 
Line Graph: Live Births 



in the United States 



THE NATION'S 
REPORT 



CARD 



raep 

m 



Grade 12 


49(1.5) 


Male 


52 (2.1) 


Female 


46(1.9) 


White 


53(1.8) 


Black 


31 (2.8) 


Hispanic 


29 (3.8) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 U.S. History Assessment. 



Nearly half of the twelfth-grade students correctly chose option C. It appears that 
twelfth-grade students were able to read a line graph that plots birthrate by year and also 
knew, at least approximately, the year the Second World War ended. A substantial 
proportion of the students (29 percent) chose option D. While we cannot be sure of the 
reasons for this choice, it seems likely that this option would have been attractive for 
students who were not clear on the dates associated with the Korean War or the First 
World War. 
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Explaining Graphic Data 

In addition to extracting information and identifying trends or patterns, students also were 
presented with questions that asked them to interpret or explain the information or trends 
they observed in a graph. Students were expected to apply outside historical or 
geographical knowledge in their explanations. 

Sample Geography Question 

Grades 8 and 12 — Bar Graph: U.S. Travel to Europe 

The following question concerns travel of U.S. citizens to Europe between 1948 
and 1991. 




This two-part short constructed-response question required students to read the 
bar graph, which plots number of citizens traveling by year; identify the dominant trend in 
travel from 1948 to 1991; and explain reasons for the trend using their outside 
knowledge. The expectations for a “Complete” answer included the correct identification 
of the major trend (travel to Europe increased over time) and two appropriate reasons for 
the trend. Acceptable reasons included improvements in travel, such as air travel being 
more affordable, faster, and safer; increases in tourism because of more leisure time and 
more consumer affluence; and internationalization of the economy leading to more 
business and political travel. A “Partial” response identified the increasing trend but gave 
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no reasons for the trend or provided one reason only. Partial credit also was given to 
responses that provided two appropriate reasons, but failed to explicitly identify the trend 
as instructed. An “Inappropriate” response did not correctly and explicitly identify the 
major trend, although, in some instances, it may have included one appropriate reason for 
the trend. Sample student responses in each of these three score categories are presented 
below: 



Sample “Complete” Response 



9. What is the dominant trend shown in the graph? 

JJL Xit/rvl Mr — 

tHi t~ lA» cdyn qasl 

ala ~ b> p yp — Wytw 



Give two major reasons for the trend shown. 

{Oryg. 

fcoi-a &)£* jb* orv 

U d ki ptfpfit LrptO 

to 6 \ jb CuAapg. 
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Sample “Partial” Response 



9. What is the dominant trend shown in the graph? 

rLfinnr\ir^oifnjt> in&nruoU afoaujjru Jmnl-ho , 

Ik Jjko 

cyxj Yxrx J3P £xU\A-fi5L> 3rvj0i^ 

^EAJLPJfrld S^PrvTv -> V\5_L 

Give two major reasons for the trend shown. 

nrruyji ^ n fla^ flryvk, j&SL, frvflfflA. 

QjtCl l&jxkj orf^S- r±X&dbL-Xifdirk 
lyfsWio , futiMpSL, qjwdL 
— finiag^Q — wrWlfllL £ji9J&f3L> Inna 19<$. 



Sample “Inappropriate” Response 

9. What is the dominant trend shown in the graph? 

JI± . < ^irY>vjAS y»^ rAQAjt OAi^di Sfqfre^ 

LLiuejQS ^rwv^gV.Vft fuK>pg- ftmW 

i(Pat- yiACg. iQifi? 

Give two major reasons for the trend shown. 

^apufohon and inq£>S pp^ 
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Information on student performance on this question is presented in Table 4.7 
for both grades 8 and 12. This question appears to have been quite difficult for students in 
the eighth grade. Seventeen percent of students chose not to respond to this question at 
all. More than half provided a response that was scored as “Inappropriate.” These eighth- 
grade students were not able to look at the graph and articulate the fact that the dominant 
trend was one of increasing travel, let alone provide reasonable explanations for this 
trend. Fewer than a tenth of the students were able to provide a complete response by 
correctly characterizing the trend and providing two appropriate reasons. 

Twelfth graders performed better on this question than eighth graders. One 
quarter provided “Complete” responses. On the other hand, a little over a third still gave 
“Inappropriate” responses, while an additional 7 percent failed to respond at all. 



Table 4.7 


Score Percentages for Bar Graph: 
U.S. Travel to Europe 


THE M 
REPORT 
CARD 


lATION’S 

raep 

=*; 


Complete Partial Inappropriate Omit 



Grade 8 


9 (0.8) 


22 (1.1) 


54(1.5) 


17(1.2) 


Male 


7 (0.8) 


21 (1.7) 


55(1.8) 


17(1.5) 


Female 


10(1.4) 


22(1.6) 


52 (2.1) 


16(1.6) 


White 


11 (1.D 


26(1.7) 


50(1.8) 


13(1.3) 


Black 


0 (0.4) 


11 (1.9) 


61 (3.0) 


28 (2.6) 


Hispanic 


3(1.1) 


13(2.6) 


60 (3.4) 


24 (2.9) 



Grade 12 


26(1.5) 


33(1.5) 


35 (1.4) 


7(0.7) 


Male 


25(1.8) 


32(1.9) 


36(1.7) 


7(1.0) 


Female 


27 (2.0) 


33(1.9) 


33 (2.1) 


7(0.8) 


White 


30(1.9) 


35(1.9) 


31 (1.8) 


4(0.7) 


Black 


6(1.8) 


24 (2.5) 


55 (3.3) 


15(2.5) 


Hispanic 


19(2.7) 


24 (2.8) 


44 (3.4) 


14(3.1) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Sample U.S. History Question 
Grade 12 — Table: Farm Sizes 

Like the preceding question, students were asked in this question to understand the data 
presented and provide an explanation for any trends or patterns observed. In this question, 
students were presented with a table containing quantitative data, which they were asked 
to understand and explain using outside historical information. 



AVERAGE FARM SIZE AND TOTAL NUMBER OF FARMS 


Year 


Farm Size 


Number of Farms 


1900 


150 acres 


6,250,000 


1980 


425 acres 


2,225,000 . 



7. Summarize the changes shown in the table above. 



Explain how one invention or development helped cause the changes 
you have described. 



To answer this short constructed-response question correctly, students needed to 
be able to read the table, recognize that between 1900 and 1980 the average farm size 
increased while the total number of farms decreased, and provide one reason for the 
trends in farm size and number of farms. A “Complete” response included a summary 
that indicated that there were fewer and larger farms in 1980 and clearly explained this 
change as a result of an historically accurate development. There were numerous 
acceptable inventions or developments that students could mention in explaining the 
change. These included: fertilizers that allowed individuals to farm more acres; 
machinery such as combines, which allowed people to farm large areas; declining prices 
for farm products, which drove people away from farming; the rise of agribusiness, which 
made small farming uncompetitive; natural disasters such as droughts and pestilence 
(e.g., boll weevils) which drove people off the land; high cost of machines and gasoline, 
which made small farms economically uncompetitive; rising land prices, which led small 
farmers to sell to developers; and an increase in factory jobs that drew off marginal 
farmers. 



9 Q 

^ o 



114 



Using the Tools and Resources of Geography and U.S. History 



A “Partial” response accurately summarized the data in the table but did not 
explain how an invention or development caused the changes. Responses were also 
scored as “Partial” if they did not explicitly summarize the change, but explained it as 
arising from the effects of an appropriate invention or development. An “Inappropriate” 
response did not correctly summarize the data, nor did it identify an invention or 
development that affected the change. Examples of student responses are presented 
below. 



Sample “Complete” Response 



7. Summarize the changes shown in the table above. 

r\ VKjl ^ 






Explain how one invention or development helped cause the changes 
you have described. 

fcoji. u&Q Kjl. 

W&i ^rJiA i/yiAl&Q (os n^rs jo 

Cl Iffrrrxtft*. i [oh 
OdulU I h/’y fawiS 

pi AKWir/ ulil'h ffvA tKxL/rX 
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Sample “Partial” Response 



7. Summarize the changes shown in the table above. 

^rrvJxN • • 



Explain how one invention or development helped cause the changes 
you have described. 

4Kl»v\. 






Sample “Inappropriate” Response 



7. Summarize the changes shown in the table above. 

ax a2z — i — Z&£Q- ^ I^QJ 



<2L 



Lja o^iqg /'n x/Jt Sastrl xAg ^ ^rr AS 



Explain how one invention or development helped cause the changes 
you have described. 

j/ s. ^ Qa/ “ Wr-/ 

in X e.s/aJ >V\. — ]*a / 4 WSC) 



pt&L 



-X-/ i <n r/ &rtr\ Atrdi 

cp\ae^ ILL 
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Student performance is presented in Table 4.8. About 30 percent of students 
were able to provide responses that were scored as “Complete.” They were able to 
correctly summarize the data and provide an explanation for the changes in terms of an 
invention or a development. Almost three quarters of twelfth graders received a 
“Complete” or “Partial” score, indicating that they were able at least to summarize the 
data in the table. 



Table 4.8 


Score Percentages for Table: Farm Sizes 


THE h 
REPORT 
CARD 


(ATION’S 


Complete Partial Inappropriate Omit 



Grade 12 


29{1.5) 


45 (1.6) 


22 


(1.1) 


4 (0.6) 


Male 


31 (1.8) 


44 (2.2) 


21 


(1.4) 


5(1.0) 


Female 


28(1.9) 


46 (2.0) 


23 


(1.6) 


4 (0.6) 


White 


32(1.8) 


44(1.9) 


21 


(1.3) 


3 (0.6) 


Black 


15(2.7) 


52 (3.4) 


26 


(3.7) 


7(1.7) 


Hispanic 


15(3.2) 


45 (4.8) 


32 


(4.2) 


8(1.7) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1994 U.S. History Assessment. 
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Sample Geography Question 

Grade 12 — Bar Graphs: Levels of Hydrocarbons 

In this question, students were presented with a bar graph of predicted levels of 

hydrocarbons by year, and were asked to observe a trend and provide two possible 

explanations for it. 



ESTIMATED LEVELS OF HYDROCARBONS 
IN THE GLOBAL ATMOSPHERE FROM 1980-2020 

■ _ - I 



1980 1985 1990 1995 2000 2005 2010 2015 2020 

Year 

17 . The graph shows a prediction made in 1970 of levels of hydrocarbons 
in the atmosphere. Describe the pattern indicated by the graph. 

Explain two factors that could account for the pattern shown. 



To correctly answer this extended constructed-response question, students 
needed to be able to read the graph, identify the decreasing-then-increasing trend, and 
apply outside knowledge about hydrocarbons and societal behaviors that are related to 
hydrocarbon levels. In order to receive a “Complete” score, students needed to identify 
the pattern shown in the graph and provide two appropriate factors that could account for 
the trend, one linked to a decrease in hydrocarbon emissions and one to an increase. To 
receive complete credit, the factors had to be explicitly linked to the decrease or the 
increase. Appropriate reasons for a decrease could include environmental regulations, 
cleaner auto emissions, increased use of public transportation, a slow-down in the 
economy, and a reduction in the combustion of fossil fuels. Acceptable reasons for an 
increase in hydrocarbons could include increased automobile use, industrialization of less 
developed countries, increased use of fossil fuels, an increase in the population, and an 
increase in the decomposition of garbage in landfills. 
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An “Essential” response identified the correct trends in hydrocarbon levels and 
provided one factor linked either to a decrease or an increase in hydrocarbons. A “Partial” 
response identified the trend but provided no appropriate factors; did not identify the 
correct trend but provided one factor linked correctly to decreasing or increasing levels of 
hydrocarbons; or provided one or two appropriate factors but did not directly link them to 
decreasing or increasing trends. An “Inappropriate” response did not identify the correct 
trend, nor did it provide any appropriate factors. Examples of student responses are 
provided below. 

Sample “Complete” Response 



17. The graph shows a prediction made in 1970 of levels of hydrocarbons 
in the atmosphere. Describe the pattern indicated by the graph. 

~7^fl/e y»ill be a Jeorra se. JMb[ -Mg yOXf 

2.CO Q+ Jccram w,n hi 

is. 'ihcrot£± 

Explain two factors that could account for the pattern shown. 

/>KAi.y ft KM^vA/ 4*cbr a ky a //***"<*, 

'fas wl/l JiBid. a Jr>{ k Z///M jJosJ&jJl J± 

uM oJLAl cea&ttJ. J 

rU ^cooxrbou in -/k 
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Sample “Essential” Response 



17. The graph shows a prediction made in 1970 of levels of hydrocarbons 
in the atmosphere. Describe the pattern indicated by the graph. 

Tlie. q.cA Al+j — by ak. grafK Wni^on p^ 

foa.IteiJg -Uuf /Q 8b—lax> >-f , y^Uo-*Gyl»r> >6 6 fen 

J“C-C.f3aAP_ K^-t- fror^ I QOO-IOL 0 hr tM>-Lk>ff^aoUy \ Kef Cage. j>e»ondt 

-fot or ky"<J Gta&J^S, 

r<9«a 

Explain two factors that could account for the pattern shown. 

People axxlA peg eovi^mrenfoUy rtdi 

4>r» CL. -flA^ t^/ow/y ikjzb- el^pir^S vraj// 

JLetec fer-pj-i . 



Sample “Partial” Response 



17. The graph shows a prediction made in 1970 of levels of hydrocarbons 
in the atmosphere. Describe the pattern indicated by the graph. 

JhjL+ dtr tUMfl flu/lA 


id 


LQtaaIu- dlTJ)MjAM Qa£ A/kL GMQiA^ 


r i 


E> 

j 

k 


iplain two factors that could account for the pattern shown. 


.f 
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Sample “Inappropriate” Response 



17. The graph shows a prediction made in 1970 of levels of hydrocarbons 
in the atmosphere. Describe the pattern indicated by the graph. 

yK\\ <y> 0*9 -K/yye <ye$ 

QAi- 



Explain two factors that could account for the pattern shown. 

(Vng agd-rloq [ft th^ r 

\ J j 



Student performance on this question is presented in Table 4.9. Sixty-five 
percent of twelfth graders received a score of at least “Partial,” indicating that they could 
identify the decreasing-then-increasing trend in the levels of hydrocarbons. However, 
only a very small percentage of students could provide one or two factors that could 
account for the pattern shown in the graph. With regard to their presentation of 
explanatory factors, many students appeared to understand that hydrocarbons had to do 
with air pollution, but they were not able to craft reasonable causal explanations for why 
these levels would rise or fall during the next half century. 



Table 4.9 


Score Percentages for Bar Graph: Levels 


THE h 
REPORT 


lATION’S 

isaen 




off Hydrocarbons 


CARD 


4 





Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 12 


1 (0.3) 


8 (0.9) 


56 


(1.4) 


27(1.6) 


8 (0.9) 


Male 


2 (0.4) 


10(1.4) 


51 


(2.5) 


29 (2.4) 


9(1.3) 


Female 


1 (0.3) 


7 (0.9) 


61 


(1.8) 


24(1.9) 


7 (0.9) 


White 


2 (0.3) 


10(1.1) 


59 


(1.8) 


24(1.9) 


6 (0.8) 


Black 


0 (0.0) 


3(1.1) 


44 


(3.2) 


39 (3.5) 


14(2.6) 


Hispanic 


0 (0.0) 


3(1.1) 


45 


(4.1) 


34 (4.1) 


18(4.2) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Constructing Charts 

It is important for students not only to be able to read from already constructed graphs 
and tables, but to also understand how to take data and present them in various graphical 
formats. 

Sample Geography Question 

Grades 8 and 12 — Pie Chart: Annual Precipitation for Lakeside 

The following question required students to construct a pie chart from data presented in a 

table. 



AVERAGE ANNUAL PRECIPITATION FOR LAKESIDE 



Season 


Average inches 


Percent of Total 


Spring 


5.0 


25 


Summer 


7.0 


35 


Fall 


4.0 


20 


Winter 


4.0 


20 


Total 


20.0 


100 



16. Use the information in the table above to construct a pie chart on the figure 
below. Be sure to label all information. You may use your ruler to draw the 
chart. 




1 



o i 
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In order to do this task correctly, students needed to be able to read the table; to 
know that pie charts are used to represent percentages; and to know that the whole pie 
represents 100 percent. Students also needed to know that a correctly constructed pie 
chart includes an accurate representation of the percentages as well as labels and other 
information to identify what is being represented on the chart. 

For this short constructed-response question, a “Complete” drawing correctly 
charted the percentage of rainfall for the four seasons and correctly labeled the parts of. 
the pie with the names of the four seasons. A “Partial” score was given to responses that 
correctly charted the percentages of one to three seasons, divided up the chart correctly 
but did not label the chart by season names, or labeled the chart by inches only. An 
“Inappropriate” response did not properly chart any of the information; it failed to divide 
the circle into pieces corresponding to the information from the table and did not 
correctly label the chart. Sample student responses are provided below. 

Sample “Complete” Response 



16. Use the information in the table above to construct a pie chart on the figure 
below. Be sure to label all information. You may use your ruler to draw the 
chart. 
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Sample “Partial” Response 



16. Use the information in the table above to construct a pie chart on the figure 
below. Be sure to label all information. You may use your ruler to draw the 
chart. 




Sample “Inappropriate” Response 




O 

ERIC 
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Information on eighth- and twelfth-grade student performance on this question is 
presented in Table 4.10. Students at both grade levels performed relatively well on this 
question. Many students at both grade levels were able to divide a pie chart into 
appropriately sized segments and knew how to correctly label the segments. One of the 
main reasons that responses received a “Partial” rating was that students did not label, or 
incorrectly labeled, their charts. Students providing “Inappropriate” responses appeared 
not to understand how pie charts are constructed. As expected, twelfth graders were more 
likely than eighth graders to provide a “Complete” response and less likely to produce an 
“Inappropriate” response. 



Table 4.10 


Score Percentages for Pie Chart: Annual 
Precipitation for Lakeside 


THE h 
REPORT 
CARD 


lATlON’S 

raep 


i- 


Complete Partial Inappropriate Omit 



Grade 8 


45(1.5) 


23(1.4) 


32(1.5) 


1 (0.3) 


Male 


45 (2.0) 


22(1.8) 


32 (2.0) 


1 (0.4) 


Female 


44(1.9) 


24(1.7) 


32(1.9) 


1 (0.3) 


White 


52(1.9) 


23(1.6) 


25(1.6) 


1 (0.3) 


Black 


13(2.7) 


24 (3.2) 


61 (3.7) 


2(1.0) 


Hispanic 


36 (3.9) 


22 (3.4) 


41 (3.6) 


2(1.0) 



Grade 12 


66(1.6) 


17 (0.9) 


17(1.2) 


1 (0.2) 


Male 


67 (2.0) 


16(1.7) 


15(2.0) 


1 (0.4) 


Female 


64(2.1) 


17(1.6) 


19(1.5) 


0(0.1) 


White 


69 (2.0) 


16(1.2) 


14(1.5) 


1 (0.3) 


Black 


41 (3.6) 


21 (2.3) 


36 (3.2) 


1 (0.7) 


Hispanic 


54 (4.3) 


21 (3.6) 


25 (2.7) 


0 (0.0) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 



Summary 

The difficulty of questions involving data in graph's, charts, and tables is a function both 
of the stimuli and the other features of the questions, such as the outside knowledge 
students need in order to answer the question and the level of inference or thinking 
students are asked to apply in answering the question. Many of the students at each grade 
level appeared able to read and understand data presented in simple graphs and tables. 
Their performance, however, suggested that they had difficulty when asked to explain the 
data or information displayed in the graphic. 
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Chapter 5 



Working with 

Photographs and Historical Art 

Both history and geography are fields in which practitioners interpret visual images, such 
as photographs or various forms of artistic expression. Photographs can help students of 
geography learn to recognize characteristics of places. Because they can show physical 
features (such as land forms) or conditions (such as climate) of an area or region, 
photographs can be more useful than descriptive text for instruction or investigation. 
Photographs are also useful for showing students different uses of land and different ways 
that people interact with their environment. 

Photographs and art from different historical periods are valuable instructional 
tools for creating connections between peoples and events across time. As part of the 
movement in secondary school history instruction to incorporate primary source 
materials, teachers are using non-textual sources to expose students to different aspects of 
the past, and to develop critical viewing skills. 1 



Photograph and Art Questions in the 1994 Geography and 
U.S. History Assessments 

The 1994 NAEP U.S. history and geography assessments both contained questions using 
photographs and art as a stimulus. The U.S. history assessment contained many exercises 
that used a variety of artworks such as paintings, murals, totem poles, cartoons, posters, 
magazine covers, and sketches, as stimulus material. Because of the greater variety and 
frequency of use of these visual stimuli in the history assessment, this chapter will focus 
on the photography and art used in questions from the U.S. history assessment only. 

When working with photographs and art in the U.S. history assessment, students were 
asked to do one or more of the following: 

• show basic comprehension of the photograph or work of art 

• relate the artwork to historical conditions or events 

• use the artwork as an interpretive tool 

• compare and contrast works of art 

' Payne, B. (1993). A word is worth a thousand pictures: Teaching students to think critically in a culture of 
images. Social Studies Review , 32 (3), 38-43. 
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Frequently, two or more questions testing different skills and abilities were based 
on the same photograph or work of art. The first question often tested basic 
comprehension of the artwork, while the succeeding questions allowed students to expand 
on their knowledge or understanding by asking them to relate the art to a geographic or 
historical context or to interpret, analyze, or evaluate the artwork. 



Examples of 1994 U.S. History Photograph and Art Questions 

The types of tasks students were asked to perform were quite similar for both 
photographs and historical art, so the examples that follow will include both types of 
stimulus materials. 



Inducing Information from Photographs or Works of Art 

At all grade levels, the assessments included questions in which students were presented 
with photographs or historical art and asked to identify or describe what they saw. These 
questions tested students’ understanding of the photograph or work of art, often requiring 
them to deduce facts about individuals and the way they lived from what was shown in 
the picture. The following exercise was given to fourth graders. 
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Sample U.S. History Question 

Grade 4 — Drawing of Indian Village: How Do They Get Food? 



Questions 5-6 are about the drawing below, which shows an American 
Indian village that was located on the southeast coast of North America. 




5. Name TWO ways the people in the village could get food. 
What in the picture helped you to answer the question? 
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The exercise asked students to name two ways people in the village could get 
food, and to specify what in the picture led them to this conclusion. The exercise required 
a short constructed response. A “Complete” response noted at least two of the following: 

farming, based on the crops shown throughout the village 

hunting, based on the bows and arrows shown in the upper left-hand comer of 
the picture 

gathering of wild foods such as acorns or berries, with some attempt to justify 
based on the picture 

fishing, based on the fish shown in the center of the picture. 

A “Partial” response identified only one method of obtaining food, or identified 
two methods but did not use evidence in the picture as support. An “Inappropriate” 
response did not identify any of the relevant methods of getting food. Actual student 
responses in each of these categories are shown below. 

Sample “Complete” Response 
5. Name TWO ways the people in the village could get food. 

1. People. in ±jge. 1/iKia.jK.^ ouoolcL ro^JLj I 
v/eg e.-k*b i<°? . 



2 . rkoplo 4 K^ Vtlli'feNC) c AUSO ^/jnTCD 

JSeQpf t flu f Tex ) f Q < d e 



What in the picture helped you to answer the question? 

look ina J*rr ALk ~Thfc- 0/3<^ AKT Q 
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Sample “Partial” Response 



5. Name TWO ways the people in the village could get food. 

i. r Ati L-/ CC ¥e> 



■> q.A Pi J k / ^ LA j-oiv’CL. 



What in the picture helped you to answer the question? 

Th-e- |p I ece<s In a k p 



Sample “Inappropriate” Response 



5. Name TWO ways the people in the village could get food. 

l. Thexj QouJd. -CjeJi foot/ ^ Wo*l 

jfo Sthm g€.f~ ^ad fen — jjlgd. C 



~font«ftA 



2 . Tip ij mu Id r*dc or a — h&rs <L 

"fc^-c L 



What in the picture helped you to answer the question? 

Ikfi, pg/yJe_ heJfzeA <xnSU/&m — fAc . .. 

At 
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Student performance data are presented in Table 5.1. Overall, approximately 
97 percent of the fourth graders chose to respond, and more than one-third provided a 
•response that was scored as “Complete.” Another 54 percent received a score of “Partial.” 
Less than 10 percent received an “Inappropriate” score. This performance indicates that 
the majority of students who responded were able to describe how the people in the 
village could get food, and approximately one-third were also able to explain what 
evidence from the drawing supported their answer. This performance suggests that fourth 
graders — though they have not taken a formal U.S. history course — may have studied 
about Native Americans in their social studies classwork, and/or appear to be able to 
induce information from visual images and were able to draw on this knowledge to help 
them understand the picture. 



Table 5.1 



Score Percentages for Drawing of Indian 
Village: How Do They Get Food? 



THE NATION’S 
REPORT licawi 
CARD “ 



13 ; 





Complete 


Partial 


Inappropriate 


Omit 


Grade 4 


34(1.6) 


54(1.6) 


9 (0.9) 


4 (0.4) 


Male 


35 (2.1) 


51 (2.4) 


8(1.1) 


5 (0.7) 


Female 


33 (2.1) 


56 (2.1) 


9(1.2) 


2 (0.5) 


White 


39(1.8) 


53 (2.0) 


6(1.0) 


3 (0.5) 


Black 


22 (2.8) 


57(3.4) 


16(2.3) 


6 (0.9) 


Hispanic 


24 (4.5) 


55 (4.0) 


14(2.7) 


7(1.4) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Proqress (NAEP) 
1994 U.S. History Assessment. 



Relating Art to Historical Conditions 

Students were also presented with questions that required them to place an artwork or 
photograph in an historical context. These questions required students to recall and apply 
knowledge from their history or social studies classes. The following exercises are from 
the fourth-grade assessment. 
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Sample U.S. History Question 

Grade 4 — Ellis Island Photographs: What Do They Show? 

In this exercise students were presented with two photographs of Ellis Island and asked 
what part of history they could learn about by visiting the island. The first photograph 
simply shows an aerial view of the island and the buildings that occupy it. The second 
photograph shows a group of immigrants with their luggage and identification papers. 




Lewis Hine Memorial Collection. 



12. Look at the pictures of Ellis Island on page 8. What part of history could 
you learn about by visiting Ellis Island? 

(a) The way the United States became a new country 
CD The war the United States fought against Spain 
CD How people lived in North America before Europeans arrived 
^ Some of the people who came to the United States as immigrants 
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The student performance data are presented in Table 5.2. They show that 30 
percent of all students taking the test knew that visiting the island would allow them to 
learn about people who came to the United States as immigrants (Option D). Analysis of 
the response data shows that another 33 percent selected Option C, that they would learn 
how people lived in North America before Europeans arrived. 




Percentage Correct for Ellis Island 


THE * 
REPORT 

rARn 


lATlON’S 

ram 


Photographs: What Do They Show? 


LAnli 





Grade 4 


30(1.3) 


Male 


31 (2.0) 


Female 


29(1.9) 


White 


34(1.9) 


Black 


18(2.3) 


Hispanic 


20 (2.8) 



Standard errors of Hie estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Proqress 
(NAEP), 1 994 U.S. History Assessment. 



Sample U.S. History Question 

Grade 4 — Drawing: Transportation in United States 100 Years Ago 
In this exercise, students were shown drawings of three forms of transportation that were 
common 100 years ago (a stagecoach, a sailing ship, and a horse and rider) and asked to 
answer two questions. In the first they were required to name a form of transportation that 
is common now and explain how it differs from the forms shown in the drawings. In the 
second they were asked to extend their answer by describing how the method they named 
is better than forms used 100 years ago. In order to answer these questions, students 
needed to be able to: 1) identify the objects in the drawing; 2) identify a modem form of 
transportation; 3) analyze how a form of transportation used today is different from the 
forms shown in the sketches; and 4) evaluate how such a form of transportation is better 
than those in the sketches, which requires understanding the limitations of older forms of 
transport. 
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3. These pictures show three common forms of transportation to the United States 
100 years ago. 

Name one form of transportation that is common now in the United States, and 
explain how it is different from the forms of transportation shown above. 

Describe one way that the form of transportation you named is better than the 
forms of transportation that were used 100 years ago. 





For a “Complete” response to this extended constructed-response question, 
students were expected to correctly identify one form of modem transportation and 
describe how it is both different from and better than one of the forms of transportation 
shown in the pictures. An “Essential” response correctly identified a modem form of 
transportation and described either how it was different or how it was better. A “Partial” 
response identified a modem form of transportation but did not answer the rest of the 
question or described a difference or improvement but failed to explicitly identify a 
modem form of transport. An “Inappropriate” response failed to identify a modem form 
of transportation or describe how transportation is different or better today. 

Acceptable responses for a type of transportation and how it is different included 
the following: 

• airplane: faster, carries many people, uses gasoline, airlines are a big business 

• railroad: faster, carries many people, runs on electricity or hydrocarbon fuel, 
can be used in almost any weather 

• automobile: faster, runs on gasoline, can be used in most weather conditions 

• transport boats: larger, run on hydrocarbon fuel rather than wind, are faster 
than sailboats 

• modem forms of transportation are made of different things, like metal or 
plastic, use computers, etc. 

• modem forms of transportation have things like lights, radios, air 
conditioning, etc. 
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Acceptable responses for how transportation is better today include: 

• travel is faster 

• travel is easier/more comfortable/convenient 

• distance is a less important factor; travel takes a lot less time, so distance is 
less of an obstacle 

• travel is cheaper today for long journeys 

• travel is less dependent on variables of weather (e.g., no wind for sail, mud for 
carriages/wagons/stagecoaches to get stuck in) 

• today’s transport often carries more people than were carried in carriages, 
small sailboats 

• current forms of transport can hold more things 

Sample “Complete” Response 

f 

Name one form of transportation that is common now in the United States, and 
explain how it is different from the forms of transportation shown above. 

A ca r ' fco (h q. because. j\ 

pg\.W\g6 — 3L.Y* also -\VtoOrv fr. 

QQrrftfle. PnViXse, o* cor rod e. a. oas- \\<xs 

tnsXAe. g, powervm^ 

Describe one way that the form of transportation you named is better than the 
forms of transportation that were used 100 years ago. 

/\ Crxr — Vtaa aXlouV QQOO 

QaxJSXl Vq >Tf\g^t<L — — Qq — YV\qy\ cv C&JcxC^. 

Ay Cor ^WtirV Q gg, ^Oou u /xYV^ 

Wrses yo \\- A 
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Sample “Essential” Response 



Name one form of transportation that is common now in the United States, and 
explain how it is different from the forms of transportation shown above. 

a car a c.gr [ $ dcJYjrs pj 
hcc_£jjj ±_ 4 <3 rnvf er- 

Q £. iiz 



Describe one way that the form of transportation you named is better than the 
forms of transportation that were used 100 years ago. 

/?C/ 12 . £ 



Sample “Partial” Response 

Name one form of transportation that is common now in the United States, and 
explain how it is different from the forms of transportation shown above. 

JjA UhX* / 



Describe one way that the form of transportation you named is better than the 
forms of transportation that were used 100 years ago. 

&MML /ryyur 

JAl 
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Sample “Inappropriate” Response 



Name one form of transportation that is common now in the United States, and 
explain how it is different from the forms of transportation shown above. 



7L 



sjgQ< > . c qU 



l S ns gd in 



-ihz. United fe g . Tf d. r£fkr&ni. from fA c 

ha&b — Lza&llsz Jhjadl c>n \A)a./er.TA<^ 

horsa is dl ££sl re oh A ejLauSf. ijl can qa! 



- aca j- 



on(L. 



f 



arson. 



Describe one way that the form of transportation you named is better than the 
forms of transportation that were used 1 00 years ago. 

faib.^e.g-o.ac A ls A g.ffgr* ~W\o.n iA&. 

lea nSp o rid h' o r\ &yQ til 

Can carr^ more people ’1^ rJso A)C^ 

SpP . Ci q J ib — pa-apile . — Srvng t/mgs A&aAitgg. JO of 

& — S<? p cfc 



flaAfl 



y — P £o f Zft — 9 
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Information on student performance on this exercise is presented in Table 5.3. 
Sixty percent of fourth graders provided a response that was scored as either “Complete” 
or “Essential.” Less than 10 percent gave “Partial” responses, and approximately 30 
percent gave “Inappropriate” responses or omitted the question entirely. Students giving 
“Inappropriate” or “Partial” responses appeared to have difficulty understanding the 
question and making a distinction between different and better. Students seemed to have 
a greater understanding of what makes modem transportation different (e.g., uses 
electricity or gasoline instead of horses) than they did of what makes it better (e.g., 
faster). Morever, understanding what makes modem transportation better is a somewhat 
complex task as it requires recalling or inferring inconveniences or other drawbacks 
associated with the earlier forms of transportation and then contrasting these 
inconveniences with the conveniences of modem transportation. 



Table 5.3 


Score Percentages for Drawing: 
Transportation in the United States 
100 Years Ago 


THE h 
REPORT 
CARD 


lATlON’S 

raep 




Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 4 


36(1.4) 


24(1.1) 


9 (0.9) 


21 (1.0) 


10(0.8) 


Male 


39 (2.0) 


24(1.6) 


8(1.2) 


18(1.3) 


12(1.4) 


Female 


34 (2.0) 


24(1.5) 


10(1.1) 


24(1.6) 


9(1.1) 


White 


42(1.7) 


26(1.5) 


8 (0.9) 


17(1.0) 


8(1.0) 


Black 


17(1.7) 


18(2.1) 


14(2.6) 


33 (3.0) 


18(2.1) 


Hispanic 


22 (4.2) 


22 (3.0) 


9 (2.0) 


30 (3.8) 


17(2.6) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 U.S. History Assessment. 
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Sample U.S. History Question 

Grades 8 and 12 — Civil Rights Cartoon: Explain When 

In this exercise, students were asked to look at a cartoon, choose from four options the 
decade in which they believed the cartoon was drawn, and give historical evidence to 
explain why they chose the decade they did. 



Questions 16-17 refer to the cartoon below. 




There’# a long, long trail a* 
winding 



Bruce Shanks in the Buffalo News, 



16. Circle the decade in which you believe this cartoon was drawn. 



1920’s 1940’ s 1960’s 1980’s 



Citing specific historical evidence, explain why you chose the 
decade you did. 
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This question required students to understand that the cartoon depicted something 
about the progress of the Civil Rights Bill and to associate it with a time period in which 
its message would have been relevant. For this short constructed-response question, a 
“Complete” response correctly identified the 1960’s as the decade in which the cartoon 
was drawn and supported this with a specific reason, for example, Martin Luther 
King, Jr., Malcolm X, Rosa Parks, civil rights legislation, or civil rights demonstrations. 
Or, the response identified the 1980’s and gave a reasonable explanation, such as, “in the 
1980’s, the spirit of 1960’s civil rights legislation remained unfulfilled.” A “Partial” 
response identified either the 1960’s or the 1980’s as the decade, but gave only a vague 
explanation for why the cartoon might have been drawn in the decade chosen. An 
“Inappropriate” response did not identify the 1960’s or the 1980’s as the decade, and 
either omitted the explanation or gave an inappropriate explanation. 



Sample “Complete” Response 2 




16 . Circle the decade in which you beligjie^his cartoon was drawn. 

1920’s 1940’s 1980’s 

Citing specific historical evidence, explain why you chose the 
decade you did. 

£h£AM£U it l&aka Ms. qM 
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2 The fact that the student “misdated” the passage of Civil Rights Bill by one year was not counted as a 
substantial error. 
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Sample “Inappropriate” Response 



16 . Circle the decade in which you believe this cartoon was drawn. 

QJjoJ) 1940’s 1960’s 1980’s 

Citing specific historical evidence, explain why you chose the 
decade you did. 

^ LQ. kddsrkj 

up cl — biycas OaAy^ 6&L hdJlh \ 



Information on student performance on this exercise is presented in Table 5.4. 
This exercise was relatively difficult for eighth graders. Although 99 percent of eighth- 
grade students responded to this question, only 16 percent provided responses scored as 
“Complete,” and only another 10 percent received a score of “Partial.” More than 70 
percent could neither identify the 1960’s or 1980’s as the decade nor explain why the 
cartoon would have been drawn in these decades. 



Table 5.4 



Score Percentages for Civil Rights 
Cartoon: Explain When 



THE h 
REPORT 
CARD 


lATION’S 









Complete 


Partial 


Inappropriate 


Omit 


Grade 8 


16(1.1) 


10(1.0) 


73(1.4) 


1 (0.3) 


Male 


15(1.5) 


11 (1.2) 


72(1.7) 


2 (0.6) 


Female 


17(1.8) 


9(1.2) 


73(2.1) 


1 (0.3) 


White 


16(1.5) 


11 (1.3) 


72(1.9) 


1 (0.4) 


Black 


19(2.3) 


8(1.4) 


69 (2.7) 


3(1.2) 


Hispanic 


12(2.5) 


8(1.9) 


79 (2.5) 


1 (0.5) 



Grade 12 

Male 

Female 

White 

Black 

Hispanic 



35(1.9) 


12(1.0) 


53(1.9) 


1 (0.2) 


36 (2.4) 


13(1.5) 


50 (2.3) 


1 (0.4) 


33 (2.2) 


11 (1.1) 


56 (2.4) 


1 (0.2) 


36 (2.3) 


12(1.1) 


51 (2.3) 


1 (0.3) 


35 (3.0) 


15(3.0) 


49 (3.9) 


1 (0.5) 


20 (3.2) 


6(1.3) 


73 (2.8) 


1 (0.6) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Proaress (NAEPI 
1 994 U.S. History Assessment. 
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As expected, twelfth-grade students performed better on this exercise than the 
eighth graders, although the proportion that were able to provide “Complete” or “Partial” 
responses was still less than half. Thirty-five percent of twelfth-grade responses were 
scored as “Complete.” Another 12 percent received partial credit. 

Using Art as an Interpretive Tool 

Another set of questions on the U.S. history assessment required students to use the 
photograph or artwork as an interpretive tool. For example, students might be asked to 
interpret, analyze, or evaluate public responses to an event or individual depicted in a 
cartoon, magazine cover, or painting. Recall of factual knowledge was usually required. 

As noted earlier in the chapter, frequently two or more questions testing different 
skills and abilities were based on the same photograph or artwork. The following question 
assessing the ability of students to use art as an interpretive tool was based on the civil 
rights cartoon shown in the previous example and was given to both eighth and twelfth 
graders. 

Sample U.S. History Question 

Grades 8 and 12 — Civil Rights Cartoon: What Message? 



17. What is the main message of this cartoon? 



This question asked students to describe the main message of the cartoon. 

Students needed to analyze the cartoon, apply their historical knowledge of the progress 
of civil rights, and write a short constructed response describing why there has been a gap 
or “long trail” between passing legislation and resolving civil rights issues. A “Complete” 
response correctly identified the main message of the cartoon as being that the passing of 
civil rights laws did not mean that civil rights issues were resolved, and that social, 
economic, and political changes also were necessary. A “Partial” response demonstrated 
understanding that the cartoon was about problems associated with the Civil Rights Bill 
but did not explain the cartoon in a specific and precise fashion. An “Inappropriate” 
response did not indicate that the cartoon referred to problems associated with the civil 
rights movement in the United States. 

Actual student responses in each of these categories are shown below. 
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Sample “Complete” Response 




Sample “Partial” Response 



17. What is the main message of this cartoon? 

• ~*~*^*-* ■ ^ 

J . I« i t* A &£. — <• ^ t>- a 

g'TM. UOAf flip ^ cj6» 

« y * o*~d ,jL> , — j> 

$ 

CmA&laa 

Sample “Inappropriate” Response 
17; What is the main message of this cartoon? 

Jo flkf- «&/ i (hAL-Ofi 'i2«^/bfaL 

— =te U-r+U.AJ UrLU^MU. COM 

fUA/ti Oja tX i uAULAfiAtOLVtei 

. Jo dCTAMlltMAjCLatl LPltfl 

$ Ma i a wumklui • 
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Information on student performance on this exercise is presented in Table 5.5. 

This question was difficult for students at both grade levels. Although approximately 95 
percent of students at each grade level responded, only 2 percent of eighth-grade 
responses and 15 percent of twelfth-grade responses were scored as “Complete.” Another 
10 percent of eighth-grade and 22 percent of twelfth-grade responses were given partial 
credit, but the majority of responses at both grade levels were considered “Inappropriate.” 
In other words, these responses did not correctly identify the message of the cartoon as 
referring to problems in the civil rights movement. 

These performance data seem to indicate a lack of familiarity with the Civil 
Rights Bill and the progress of civil rights in the latter part of the twentieth century and/or 
difficulty in interpreting the content of political cartoons. Possibly also causing 
difficulties for students was the sophistication of the cartoon and its vocabulary. 
“Fulfillment in spirit,” for example, may have been an unfamiliar concept to many 
students. 



Score Percentages for Civil Rights 


THE h 
REPORT 
CARD 


lATiON’S 

i\aep 


Cartoon: What Message? 









Complete 


Partial 


Inappropriate 


Omit 


Grade 8 


2 (0.4) 


10 (0.9) 


82 (1.1) 


6(0.7) 


Male 


2 (0.6) 


9(1.0) 


83(1.4) 


7 (0.9) 


Female 


2 (0.5) 


12(1.4) 


81 (1.8) 


5(1.1) 


White 


3 (0.5) 


10(1.1) 


81 (1.4) 


6 (0.9) 


Black 


2 (0.8) 


10(1.8) 


85 (2.3) 


4(1.4) 


Hispanic 


1 (0.7) 


12(2.2) 


78 (2.6) 


9(2.1) 


Grade 12 


15(1.3) 


22 (1.2) 


59(1.5) 


4 (0.6) 


Male 


15(1.6) 


23(1.7) 


57 (2.0) 


5(1.1) 


Female 


15(1.9) 


22(1.7) 


62(1.8) 


2 (0.5) 


White 


18(1.7) 


24(1.5) 


55(1.9) 


4(0.7) 


Black 


6(1.6) 


16(2.5) 


75 (3.0) 


4(1.5) 


Hispanic 


7(1.9) 


16(3.4) 


74 (4.3) 


3(1.2) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 994 U.S. History Assessment. 
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Sample U.S. History Question 

Grade 12 — Cartoon: Why Is the U.S. Upset? 

In this question, students were asked to look at a newspaper headline and related cartoon 
and explain why people in the United States were upset by the event portrayed by these 
two artifacts. 



14. Give two reasons why many people in the United States were upset by the 
event shown in the cartoon and the headline. 



This question required students to understand the cartoon and headline, place 
them in an historical context, and based on that context, understand why people in the 
United States were upset by the events portrayed. Although the gist of public attitude 
could be inferred from the cartoon, students needed to understand at least some of the 
circumstances surrounding the groundbreaking Soviet satellite launch in order to provide 
two reasons to explain why many Americans were distressed. Students were required to 
write a short constructed response. Correct responses could include fear of being 
technologically behind in a nuclear world, the Cold War, the possibility that the 
Communist system was out-thinking the American system, concern about American 
inferiority, fear that Soviets were better educated, concern that the U.S. was not first in 
space, fear of the Russians, concerns about communism versus capitalism, or other 
concerns that the satellite launch gave the Russians a strategic advantage. A “Complete” 
response gave two reasons for U.S. concern, while a “Partial” response gave only one 
relevant reason why people were upset, and an “Inappropriate” response gave none. It was 
not necessary for students to mention Sputnik by name in order to get credit. 

Actual student responses in each of these categories are shown on the next page. 



SOVIETS LAUNCH FIRST MAN-MADE SATELLITE INTO ORBIT 




Frank Williams in the Detroit Free Press 
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Sample “Complete” Response 



14. Give two reasons why many people in the United States were upset by the 
events shown in the cartoon and the headline. 

*T k*. p QQfiC, t*v4> 

'I'Ko^lw^’ of* \ •Ce.rs'a** » TU o ».n<f b> 8 ~f 

*fl pi VCft** 8 UmS_- vi+A 

KUuj^^ 'h) douj tr>'fL < u<rt'~hj jj^; ^ 



Sample “Partial” Response 

14. Give two reasons why many people in the United States were upset by the 
events shown in the cartoon and the headline. 

/ fyo/rxlc ulW-C &UA-44 <*Y L* 

SC*<*\ CL Xt/xVO aJvLAtef o£ drvU^ri cCX rj 



Sample “Inappropriate” Response 



14. Give two reasons why many people in the United States were upset by the 
events shown in the cartoon and the headline. 

V<£-€? / j- Pu+Afc he&AA- 
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Information on student performance on this exercise is presented in Table 5.6. 
Overall, one-quarter of twelfth-grade students provided a response that was scored as 
“Complete,” and nearly 50 percent received a score of “Partial.” This performance 
indicates that many students were able to identify from the cartoon the event being 
depicted, recall this event in American history, and understand the response of the public 
at the time. 



Table 5.6 


Score Percentages for Cartoon: 
Why Is the U.S. Upset? 


THE * 
REPORT 
CARD 


lATION'S 

rcaep 

-*1 




Complete Partial Inappropriate Omit 



Grade 12 


25 (1.3) 


46(1.4) 


18(1.1) 


12(1.0) 


Male 


26(1.5) 


51 (1.9) 


14(1.4) 


10(1.2) 


Female 


23(1.8) 


42(1.8) 


22(1.8) 


13(1.4) 


White 


28(1.6) 


48(1.7) 


15(1.2) 


9(1.0) 


Black 


6(1.6) 


42 (3.2) 


30 (3.1) 


22 (2.6) 


Hispanic 


19(3.0) 


34 (4.0) 


28 (5.0) 


19(3.6) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 1 994 U.S. History Assessment. 
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Sample U.S. History Question 

Grades 8 and 12 — Magazine Cover: American Indians 
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12. Look at the magazine cover. What historical events would have led this 
question and picture to appear on the cover of a popular magazine in 1876? 

What attitudes displayed toward American Indians by other Americans 
are suggested by this magazine cover? 



In this extended constructed-response question students were presented with a 
magazine cover and asked two questions. The first questions asked what historical events 
would have led to this magazine cover appearing when it did, and the second question 
asked what attitudes toward American Indians were suggested by the magazine cover. 

A “Complete” response answered both parts of the question and showed an 
understanding of the problems expressed by the title “What shall we do with our 
Indians?” and the photograph of Sioux leaders. Citing specific detail, students may have 
explained that the attitudes of other Americans toward the Indians ranged from sympathy 
to paternalism to hostility. An “Essential” response answered both parts of the question, 
but provided only a general statement without any specifics for one part. A “Partial” 
response correctly answered only one part of the question or gave two answers that were 
not incorrect but were very vague. An “Inappropriate” response did not correctly answer 
either part of the question. 

Acceptable responses for why the cover would have appeared included: 

• Debates about rights to Western lands. As more people moved west to lands 
they felt they had a right to as Americans, they often seized Indian land 
illegally. Sioux Indians and other Western tribes reacted with fear and anger 
because they were losing their lands and ways of life. They often attacked 
pioneers and settlements. 

• A series of battles between Sioux and American forces had led to the 
placement of the Sioux on small reservations, usually on the poorest land. 

• Settlers and hunters slaughtered buffalo for food and sport, depriving Sioux 
Indians and other Western tribes of a source of food. 

• The discovery of gold in the Black Dakota Hills in the mid- 1 870’ s brought 
miners to Indian territories, further threatening Indian lands. 

• A terrible battle was fought between desperate Sioux Indians and American 
forces under General Custer, leading to a temporary, but severe, American 
defeat (Battle of Little Bighorn). 
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• Placement of Western Indians on reservations administered by the Bureau of 
Indian Affairs curtailed traditional Indian ways of life and often submitted 
Indians to the cruelty and incompetence of untrained government officials. It 
was clear to some Americans that reservations were not an easy or adequate 
solution. 

• Indian wars (anything warlike - rebellion, reprisals, killings, fighting). 

• Geronimo battles. 

Acceptable responses for attitudes displayed toward American Indians by other 
Americans included: 

• strong, negative emotions such as hate or wanting to get rid of Indians 

• feelings that Indians should be deprived of power, so the country can be 
settled peacefully and resources in the West that are necessary for industry can 
be developed 

• feelings that Indians should be placed on reservations in order to enable the 
peaceful White settlement and/or Indian survival 

• feelings that Indians should be assimilated into White society, as that will 
enable them to live peacefully with other Americans and/or learn a “superior” 
way of life 

• concern that Indians have been consistently mistreated by Whites and should 
be treated fairly 

• belief that the fate of Indians is a problem for White Americans to solve 

• feelings that Indians are noble, noble savages, or tragic figures 

• feelings that Indians are picturesque, exotic, admirable 

Sample student responses are shown below. 
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Sample “Complete” Response 



12. Look at the magazine cover. What historical events would have led this 
question and picture to appear on the cover of a popular magazine in 1876? 

— J= neLgA a.nd */fl 

af- ir a nitres' fa r/nPs'S 

A-/VY Xndbtk/is o n re 

What attitudes displayed toward American Indians by other Americans 
are suggested by this magazine cover? 

-/-AeV wo. re fi 

Aiuzl&X\ &=jiAj£gjL 

Ofrdjhd Q=od iMO.r'** 

Q- r\LL\S.<Krc& — is ~PfQrih'€/' ~£-a r'm < f~y mir) 

4& til £jZEL <g g cl A Qfr- Eg ti Q 

/2jAti 1 /t^p . y — j^Jin y 



Sample “Essential” Response 



12. Look at the magazine cover. What historical events would have led this 
question and picture to appear on the cover of a popular magazine in 1876? 



ion ana picture to appear on the cover of a 

fcfo ? tihru 



Dhm [bvKuiiGm^ t^prt 

cwL £}&& all d. IM md Mn 

u ' 1 

What attitudes displayed toward American Indians by other Americans 
are suggested by this magazine cover? 



-Lk/iMilAr&L k> blnf 

tkm M Qmmd 

-oihAt&l IhmjkeL il&JiAfllimd 

tii&U flkkL oifavsi & IwHitA dtill/l 
£^dboj . 



152 



BEST COPY AVAILABLE 



Usinfi 11 w and Resources of Geography and U.S. History 





Sample “Partial” Response 



12. Look at the magazine cover. What historical events would have led this 
question and picture to appear on the cover of a popular magazine in 1876? 

What attitudes displayed toward American Indians by other Americans 
are suggested by this magazine cover? 



Sample “Inappropriate” Response 



12. Look at the magazine cover. What historical events would have led this 
question and picture to appear on the cover of a popular magazine in 1876? 

T K&, H r\cXV>ni> rMt-N/fc rrvxsr-^- \Vvix.rv l-ta. 

\£>w\e^ VYVx/7 ftorv t <jL4r W— aj£. 

ttera u)er^ 4o 

What attitudes displayed toward American Indians by other Americans 
are suggested by this magazine cover? 

TVi&Ll \rA \ Cx.<Y5 Q jg WE- W 

LhWV-e. marv a at h-Uq. toKik. 

SKxf\ aL£ y <£nj airL^ n j bd.rOu;s<2! 



o(*~ &t uQjdz V-Wd iVvA^xn^ 

are WiW. 
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Information on student performance in grades 8 and 12 is presented in Table 5.7. 
The majority of students at both grades were able to provide at least a “Partial” response 
to this question. Approximately one-fourth of eighth-grade students provided a response 
scored as “Complete” or “Essential,” demonstrating an ability to apply their historical 
knowledge of the problems between American Indians and Whites in the 1870’s. Another 
39 percent were able to partially answer the question. 



Score Percentages for 


THE l 
REPORT 
CARD 


lATlON’S 

rasp 


Magazine Cover: American Indians 









Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 8 


8 (0.6) 


18 (1.0) 


39(1.4) 


24(1.2) 


12(1.1) 


Male 


7 (0.8) 


18(1.6) 


35(1.8) 


24(1.6) 


17(1.7) 


Female 


9(1.0) 


18(1.3) 


42(2.1) 


24(1.7) 


7 (0.8) 


White 


10(0.9) 


20(1.2) 


41 (1.6) 


21 (1.4) 


9(1.1) 


Black 


3(1.2) 


10(2.2) 


30 (3.6) 


35 (3.8) 


23 (3.0) 


Hispanic 


2 (0.8) 


15(2.3) 


36 (3.6) 


30 (2.7) 


18(2.2) 


Grade 12 


21 (1.3) 


26 (1.3) 


28 (1.4) 


13(0.9) 


12(1.0) 


Male 


20(1.9) 


28(1.7) 


28(1.9) 


11 (1.4) 


13(1.5) 


Female 


22(1.7) 


24(1.8) 


28 (2.0) 


16(1.5) 


10(1.3) 


White 


25(1.6) 


29(1.6) 


27(1.9) 


10(1.0) 


10(1.2) 


Black 


9(1.7) 


17(2.7) 


34 (3.0) 


24 (3.0) 


17(2.8) 


Hispanic 


11 (19) 


18(3.1) 


27 (2.4) 


26 (2.7) 


19(2.9) 


Standard errors of the estimated 
SOURCE: National Center for Ec 


percentages appear in parentheses. 

fucation Statistics. National Assessment of Educational Proaress (NAEP), 


1994 U.S. History Assessment. 
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Comparing and Contrasting Works of Art 

In addition to being asked to demonstrate basic comprehension of a photograph or work 
of art, to place art in an historical context, and to use art as an interpretive tool, students 
sometimes were asked to look at two photographs or works of art about a single subject 
and answer questions that required them to identify commonalities and/or differences 
between the two. 

Sample U.S. History Question 

Grades 8 and 12 — Paintings: George Washington 

The following exercise is an example of a question that required students to compare two 
works of art. It was administered to both eighth- and twelfth-grade students. Students 
were presented with pictures of two paintings of George Washington and asked a series 
of three questions. The first question asked students to describe the intent behind 
paintings of this type. To respond correctly, students had to study the two paintings of 
quite different styles and decide what similar theme was being depicted by both of them. 



1G3 



Using the Tools and Resources of Geography and U.S. History 



155 




The Metropolitan Museum of Art, Gift of Col. and Mrs. Edgar William Garbisch, 1963 . 



3. Early American paintings such as these two of George Washington 
tended to 

(3) show Washington’s interest in science 
G) show life in a realistic fashion 
9 glorify American political figures 
Cfi) celebrate Washington’s humble origins 



1C 
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Table 5.8 shows student performance on this question. This question was 
relatively easy at both grade levels. Sixty percent of eighth graders answered this question 
correctly, as did 83 percent of twelfth graders. Better performance by twelfth graders is to 
be expected because of their greater exposure to U.S. history courses and to the general 
concept of artistic intent. 



Table 5.8 



Percentage Correct 
for Paintings: 
George Washington 



THE NA TION’S 
REPORT 



CARO 



raep 

s 





Grade 8 


Grade 12 


Total 


60(1.4) 


83(1.0) 


Male 


62(1.9) 


83(1.4) 


Female 


57 (2.0) 


82(1.6) 


White 


64(1.7) 


88(1.2) 


Black 


45 (2.5) 


62 (2.5) 


Hispanic 


52 (3.5) 


74 (3.2) 


Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment 


of Educational Progress (NAEP), 1 994 U.S. History Assessment. 
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Summary 

Photographs and artwork are useful for conveying complex visual information and for 
communicating the ideas and sentiments associated with different periods in history. In 
the 1994 NAEP history assessment, photographs and artworks were used primarily to 
convey a sense of the culture and attitudes of the past. 

Students were required to show basic comprehension, relate the photograph or 
work of art to historical conditions or events, use the artwork as an interpretive tool, and 
compare and contrast various pictorial representations. As in other parts of the 
assessment, the difficulty of the questions increased as students were asked for more 
inference and interpretation. 

Particularly by grade 12, however, many students were able to understand artistic 
intent as well as the literal meaning of what they were seeing. Here, as elsewhere, 
questions that required the students to formulate explanations in their own words were 
among the more difficult. 
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Chapter 6 



Working with Atlases 

Atlases are a central tool in the study and practice of geography. These bound collections 
of maps, tables, charts, and other information represent an essential reference source for 
students. Learning to use an atlas is an important geographic skill; the variety of maps and 
other forms of information in atlases can assist students in the visualization of space and 
can further their understanding of complex social and physical data about geographic 
areas. 

Overall, one block of exercises at each grade of the 1994 NAEP geography 
assessment involved use of the Nystrom Classroom Atlas. One block of 14 questions was 
given at grades 4 and 8, while a second block of 15 questions was given at grade 12. The 
questions required that students analyze a variety of maps, including physical, political, 
population, land use, landscape, climate, and time zone maps, as well as use graphic and 
text information. The questions tested a variety of skills. Students usually were referred to 
a page or group of pages in the atlas and required to observe and extract information 
presented on different kinds of maps or in graphs, charts, or tables, or to give 
explanations for patterns apparent on the maps or in the other data. 




167 



Using the Tools and Resources of Geography and U.S. History 



159 





In the 1994 NAEP geography assessment, atlases allowed for the measurement 
of a number of skills that are not usually assessed in large-scale surveys. In addition to 
general map-reading skills and the ability to analyze and evaluate information presented 
on maps and in graphic formats, the assessment measured specific skills related to atlas 
use such as the ability to: 

• Use reference materials. 

• Work with color maps. 

• Work with a wide variety of map types, including political, physical, 
population, and special-purpose maps. 

• Synthesize information from more than one map. 



Examples of 1994 NAEP Geography Atlas Questions 

Although the general skill being discussed in this chapter is the use of an atlas, the 
difficulty of the questions in the geography assessment was not defined solely by having 
to use an atlas to locate information. Once they had located the information in the atlas, 
students needed to employ a number of skills to interpret and understand it. They needed 
to extract information from the different types of maps, interpret the information found on 
the maps or in the charts in an atlas, compare different types of maps, and provide 
explanations for patterns observed in the maps. Some of the atlas questions also required 
students to use outside knowledge to respond correctly. 



Using an Atlas 

One or two questions at each grade level directly assessed students’ understanding of how 
to use an atlas to find information either by asking them where something would be 
found, or by directing them to find a certain type of map (without giving the page 
number), and then asking a question about information on the map. At the fourth- and 
eighth-grade levels, students were asked where to find information in the atlas. At the 
twelfth-grade level, the exercises were more complex and required students to answer 
questions about the information after they had located it. 
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Sample Geography Question 

Grades 4 and 8 — Atlas: How to Find Dakar 

The following question was given to both fourth and eighth graders. 



If you wanted to find out which page in the atlas had a map that showed 
the city of Dakar, where would you look? 

(a) The World Map Projections pages 

0 } The Index 

(c) The World Facts page 

(d) The Earth Notes page 



This question asked where you would look in an atlas to find out which page had 
a map showing the city of Dakar. The student performance data, which are presented in 
Table 6.1, show that the majority of the students in the nation and in the subgroups of 
interest were able to identify the index as the correct answer, although as might be 
expected, eighth-grade students performed substantially better than fourth-grade students. 
A possible explanation for the high performance of most students is that this question 
tests a general skill of how to find information and is not only specific to the use of an 
atlas. 



Table 6.1 


Percentage Correct for 
Atlas: How to Find Dakar 


THE h 
REPORT 
CARD 


lATlON’S 

raep 

=4 


Grade 4 Grade 8 



Total 


73(1.4) 


90 ( 0.7) 


Male 


73(1.6) 


90 ( 1.1) 


Female 


73(1.9) 


91 ( 1.0) 


White 


79(1.6) 


94 ( 0.7) 


Black 


58 (3.3) 


83 ( 2.8) 


Hispanic 


64 (3.1) 


79 ( 2.8) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



BEST COPY AVAILABLE 



169 



Using the Tools and Resources of Geography and U.S. History 



161 



Sample Geography Question 

Grade 12 — Political Map: Which Countries Are on Equator? 

The following question was given to twelfth graders. It is an example of a question that 
requires students to locate information in a reference source and extract information from 
the map. It directed students to find the political map of the world in the atlas, and then 
asked students to specify which group of countries was crossed by the equator. 



Find the political map of the world in the atlas. Which group of countries 
is crossed by the equator? 

(£) Mexico, Nigeria, India 

(£) Chile, Zimbabwe, Papua New Guinea 

© Venezuela, Angola, the Philippines 

• Brazil, Kenya, Indonesia 
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Table 6.2 shows student performance on this question. To respond correctly, 
students had to locate the correct map, locate the equator and trace it correctly across the 
two-page map, then check the countries listed in the response options to determine which 
option included countries crossed by the equator. The task was made somewhat easier by 
the fact that only the correct response option included any countries crossed by equator. 
The data show that the majority of twelfth-grade students were able to answer this 
question correctly. Performance of subgroups was relatively high, with no significant 
differences. 



Table 6.2 



Percentage Correct for 
Political Map: Which 
Countries on Equator? 



THE NATION’S 


REPORT 

CARD 


raep 



Grade 12 


82(1.1) 


Male 


82(1.4) 


Female 


82(1.6) 


White 


84(1.3) 


Black 


77(2.6) 


Hispanic 


75 (3.3) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: Notionol Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



Extracting Information from a Variety of Maps 

There were a number of questions in the assessment which measured students’ ability to 
extract information from maps. Minimally, students had to have some general knowledge 
of map conventions and understand the map key in order to answer these questions. In 
some cases, they were also asked to synthesize information from more than one map. 
Sometimes students were required to bring in outside knowledge to answer the questions 
posed. 
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Sample Geography Question 

Grades 4 and 8 — Physical Map: Which Region is Highest? 

In this question, students were asked to look at the physical map of Africa on a specific 
page of the atlas and identify which region in Africa had the highest elevation. The 
answer choices were phrased in terms of map directions. To answer this question, 
students needed to know the orientation of the map and understand the map key, which 
was a color-coded bar with the different colors corresponding to different elevations. 
Students had to find the color corresponding to the highest elevation on the map and 
determine which option corresponded to that location on the map. 

Look at the physical map of Africa on page 60. Which region in 
Africa has the highest elevations? 

G) Northern coast 

d) Western coast 

(c) Southwestern 

9 Eastern 
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Table 6.3 shows student performance on this question. Forty-three percent of 
fourth graders answered this question correctly. Eighth graders performed significantly 
better than fourth graders, most likely reflecting their experience with map-reading and 
coursework requiring the use of reference materials. 



fable 6.3 


Percentage Correct for Physical Map: 
Which Region Is Highest? 


THE V 
REPORT 
CARD 


lATlON’S 

rcaep 





Grade 4 


Grade 8 


Total 


43(1.4) 


65(1.5) 


Male 


46 (2.3) 


66(1.9) 


Female 


41 (1.9) 


65(1.8) 


White 


49(1.7) 


72(1.8) 


Black 


25 (2.8) 


46 (4.2) 


Hispanic 


36 (4.1) 


51 (5.2) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 


1 994 Geography Assessment. 
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Sample Geography Question 

Grades 4 and 8 — Political and Population Maps: Most Dense Population 
In this question, students were required to synthesize information from two maps. They 
were asked to look at the political map of Africa on a specific page in the atlas and then at 
the population map on a different page in order to identify which country in West Africa 
was the most densely populated. To answer this question, students needed to understand 
the map key on the population map, which was a color-coded bar with the different colors 
corresponding to different populations per square mile, and to understand the country 
boundaries on the political map. On the population map, students had to find the largest 
dark colored area, which corresponded to the highest population density; then, on the 
political map, identify the country to which that densely populated area corresponded. 



Look first at the political map of Africa on page 61, and then look 
at the population map on page 63. Which country in West Africa 
is the most densely populated? 



CD 


Liberia 


CD 


Mauritania 


(e) 


Mali 


• 


Nigeria 
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Table 6.4 shows student performance on this question. Thirty-one percent of 
fourth graders and 52 percent of eighth graders were able to correctly identify Nigeria as 
the most densely populated country in West Africa. At the fourth-grade level, Mali was 
the incorrect option most often chosen (23 percent), perhaps because Mali and Nigeria 
were the only two countries among the response options that included an area of greatest 
population density (over 250/square mile). However, the population map showed that 
most of the rest of Nigeria was moderately densely populated as well, whereas the same 
was not true for Mali. Choices of incorrect options were fairly evenly distributed at the 
eighth-grade level, suggesting that this same error was not common among eighth 
graders. 





Percentage Correct for 


THE NATION’S 


Table 6.4 


Political and Population 


REPORT 

CARD 


raep 






Maps: Most Dense Population 




-v] 







Grade 4 


Grade 8 


Total 


31 (1.2) 


52 (1.5) 


Male 


33(2.1) 


55(2.1) 


Female 


29(1.6) 


49(2.2) 


White 


32(1.6) 


54 (2.0) 


Black 


31 (3.0) 


46 (2.3) 


Hispanic 


25 (2.9) 


43 (3.7) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



Sample Geography Question 

Grade 12 — Atlas Maps: Corn Area Natural Vegetation 
In this question, twelfth-grade students were required to synthesize information from two 
maps. They were asked to look at the natural vegetation map of North America on a 
specific page in the atlas, and then at the map of com production on a different page, and 
to identify the natural vegetation of the major corn-producing areas of the United States. 



Look at the natural vegetation map of North America on page 22 of the 
atlas and the map of com production in the United States on page 3 1 . 
What is the natural vegetation of the major com-producing areas of the 
United States? 

4D Grassland and broadleaf forest 
CD Steppe and tundra 
CD Needleleaf forest and Mediterranean 

CD Desert and tropical rain forest -t o 

1 i U 
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To answer this question, students needed to understand the map keys and 
synthesize information from both maps. However, the question did not call for fine 
discriminations in that none of the other answer choices referenced vegetation types that 
were in any way proximate to the corn-growing region. Furthermore, the question could 
also be answered based on general knowledge of North American flora without specific 
reference to the atlas maps. Student performance is presented in Table 6.5 below. 



Table 6.5 



Percentage Correct for 


THE NATION’S 


Atlas Maps: Corn Area 
Natural Vegetation 


HEPOHT 

CARO 


raep 

=?: 



Grade 12 


76(1.5) 


Male 


77(2.1) 


Female 


76 (2.2) 


White 


80(1.7) 


Black 


66 (3.3) 


Hispanic 


62 (4.3) 



Standard errors of the estimated percentages appear in parentheses. 
SOURCE: National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1 994 Geography Assessment. 



Performance on this question was relatively high; more than three-quarters of 
twelfth graders correctly identified grassland and broadleaf forest as the two types of 
natural vegetation of the major corn-producing areas of the United States. 

Explaining and Interpreting Information 

In addition to being asked to find maps or other data, extract information from maps, and 
synthesize information from multiple maps in an atlas, students were also asked to engage 
in more complex analytical thinking and to provide explanations for patterns or other 
information they extracted from the maps, tables, or charts. Often this involved applying 
outside knowledge to the questions posed. 



1 
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Sample Geography Question 

Grades 4 and 8 — Map of China: Why Crowded in One Area 

Look at the population and landscape map of China on page 72 of the atlas. 
What area of the country is densely populated (crowded)? 

Give two reasons why this area of the country is densely populated. 
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This extended constructed-response question required students to examine the 
integrated population and landscape map of China and answer two questions, which were 
scored together. The first question asked which area of the country was densely 
populated, and the second asked students to give two reasons why this area was densely 
populated. For a “Complete” response, students were expected to correctly identify one of 
two areas of population concentration shown on the map (the east coastal area or Sichuan 
Basin) and to give two appropriate reasons why these areas were so heavily populated. 
Acceptable reasons were that coastal areas have flat land, suitable for building and 
agriculture, and they allow for trade with other countries; oceans and rivers are useful for 
transportation, agriculture, industry, or fishing; much of China is not suitable for 
settlement; major cities are located in the area; or the area has a lot of business, industry, 
or agriculture. 

An “Essential” response correctly identified one of the areas of population 
concentration but gave only one acceptable explanation for this settlement pattern. A 
“Partial” response correctly identified an area of population concentration but offered no 
appropriate explanation. An “Inappropriate” response neither identified an area of 
population concentration nor explained why the population would concentrate in such 
areas. Actual student responses in each of these four rating categories are included below. 



Sample “Complete” Response 



What area of the country is densely populated (crowded)? 

Tno. -Vr\CiV \S OQc^ (\QftSL\n populryVool 

',r& VViq orator r\ rnCxsV of 



Give two reasons why this area of the country is densely populated. 

rCQ^Orfc Ur»S ftrOO-’lS At>nsa\i\ OOftuWal 
tmuso Wso rr.VnnoA Ond it 

is oWo Vs iQ OOPO^Q c&o ^h. 



1£0 
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Sample “Essential” Response 



What area of the country is densely populated (crowded)? 

I U. toQvirA of 






Give two reasons why this area of the country is densely populated. 
You goo<=Cr 



Z^oo r h joJf e^rj/y. 



Sample “Partial” Response 




What area of the country is densely populated (crowded)? 

(LML4- AX\ A£l -X 

bl/l/JU ^6 JtA>b rvYlAXsvuj , 

Give two reasons why this area of the country is densely populated. 

'T/Z.sn u fULG^Qll -/Ami 



Sample “Inappropriate” Response 



What area of the country is densely populated (crowded)? 

i 



.As: 






Give two reasons why this area of the country is densely populated. 

t220HC paip fo> 
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To correctly answer the first part of the question, students had to read the map 
key and understand that solid red areas were really clusters of tiny red dots representing 
population density. For the second part of the question they needed to analyze the various 
geographic features shown on the map in light of outside knowledge, such as the 
advantages of settling near the water, to determine what might cause the settlement 
pattern. Information on student performance on this exercise is presented for both grades 
4 and 8 in Table 6.6. 



Table 6.6 


Score Percentages for Map of China: Why 
Crowded in One Area 


THE t 
REPORT 
CARD 


lATION’S 

raep 

-S; 





Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 4 


5(0.7) 


16(1.0) 


39(1.5) 


24(1.2) 


16(1.2) 


Male 


5 (0.9) 


17(1.5) 


36 (2.2) 


24(1.6) 


19(1.5) 


Female 


5(1.1) 


14(1.2) 


42 (2.0) 


25(1.5) 


13(1.6) 


White 


6 (0.9) 


17(1.4) 


42 (2.0) 


20(1.4) 


14(1.4) 


Black 


2 (0.7) 


7(1.1) 


32 (2.6) 


38 (3.2) 


22 (2.6) 


Hispanic 


3(1.1) 


12(2 .4) 


34 (4.5) 


30 (3.7) 


21 (3.4) 


Grade 8 


19(1.4) 


32(1.3) 


31 (1.2) 


15(1.2) 


3 (0.5) 


Male 


22 (2.1) 


30 (2.0) 


31 (1.7) 


14(1.7) 


3 (0.6) 


Female 


17(1.5) 


33(1.5) 


31 (1.7) 


16(1.4) 


4 (0.7) 


White 


22(1.7) 


36(1.7) 


29(1.5) 


12(1.5) 


2 (0.5) 


Black 


10(2.0) 


17(2.0) 


40 (3.2) 


23 (2.8) 


10(2.1) 


Hispanic 


15(2.2) 


25(3.3) 


36 (3.6) 


21 (2.1) 


5(1.2) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 



This exercise appears to have been difficult for fourth graders. Almost a quarter 
of the students provided a response that was scored as “Inappropriate,” and 16 percent of 
fourth graders did not respond at all. Only 21 percent were scored as having a “Complete” 
or “Essential” answer. Students may have found the question difficult because they had to 
perform three tasks: locate the highly populated areas on the map, recognize geographic 
features on the map that would encourage or prevent settlement, and identify the 
advantages of settling near certain geographic features, such as water or rich agricultural 
areas. 

As expected, eighth graders performed better on this exercise than fourth 
graders. Half of the students provided a “Complete” or “Essential” response, and only 3 
percent omitted the question entirely. Since all but 18 percent of eighth graders received a 
score of “Partial” or better, it appears that most eighth-grade students could at least read 
the map key to determine areas of population density, 
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Sample Geography Question 

Grades 4 and 8 — Bar Graph in Atlas: Use and Production of Oil 

The following exercise required fourth and eighth graders to extract and interpret 
information from a bar graph. 




Table by NY STROM Division of Herff Jones, Inc. 

Look at the bar graph on page 31. The graph shows United States 
energy production and consumption. 

Using this graph, compare the United States consumption (use) of 
oil with its production of oil. 

Name one problem that this can cause for the United States. 
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The directions instructed students to use the bar graph to compare the 
consumption of oil in the United States with its production, and then to name one 
problem that this pattern of energy consumption could cause for the United States. This 
exercise required a short constructed response. A “Complete” response correctly 
explained that the United States consumes more oil than it produces. It might have 
specified the range of actual numbers in BTUs that the U.S. produces and consumes. It 
also explained that the excess of consumption over production could lead to problems 
such as shortages, dependence on foreign countries, trade imbalance, and a high cost of 
energy. A “Partial” response correctly explained that the U.S. consumes more oil than it 
produces, or it identified a problem such as those listed above, but it did not do both. An 
“Inappropriate” response did neither. No credit was given for simply stating that the U.S. 
does not have a lot of oil. Actual student responses in each of the three rating categories 
are included below. 



Sample “Complete” Response 



"TVe. Ur>r4ggi CooSv^^n<gS ao£f_ 

d:Vi<vr\ 'fk Cy 



Name one problem that this can cause for the United States. 

Tf-P ojUssi ca <gy/ec refus<to/ -jo S£li 

1 /a r,ilj tAfi o^r\ <oi I c, 

Sample “Partial” Response 



V.^rfHit prcrtufcJA Petur-m 




_om 



W-f 






Name one problem that this can cause for the United States. 

Q.Ss ppendinc movvUn^A 
‘rn <frui i a it. \nosiVr, , /ncm.y 



84 



o 
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Sample “Inappropriate” Response 



*fA- •€ QC5~ 

\U- c,<a./*v.g — 

Name one problem that this can cause for the United States. 

Tlr’r use. €. co/g/. Co^Jd 

gx.e. t twd i\ g.y . 



Table 6.7 shows student performance on this exercise. This exercise appears to 
have been difficult for fourth graders. Almost half of the students provided 
“Inappropriate” responses, and 29 percent of the fourth graders did not respond at all. The 
performance of eighth graders was considerably better. More than 40 percent of eighth 
graders provided responses scored as “Complete,” and another 27 percent received a 
score of “Partial.” 

Many students, particularly at the eighth grade, appeared able to interpret the bar 
graph to understand that the United States was using more oil than it produced. Fourth- 
grade students, however, had particular difficulty drawing upon their outside knowledge 
to describe problems this could cause for the United States. 



Table 6.7 


Score Percentages for Bar Graph in Atlas: 


THE N 
REPORT 
CARD 


1AT10NS 

isaepl 


Use and Production of Oil 









Complete 


Partial 


Inappropriate 


Omit 


Grade 4 


5 (0.6) 


17(1.6) 


49(1.4) 


29(1.8) 


Male 


5 (0.9) 


19(2.4) 


44 (2.0) 


32(2.3) 


Female 


4 (0.8) 


15(1.8) 


54(2.1) 


27 (2.4) 


White 


6 (0.9) 


21 (2.1) 


47(1.7) 


27 (2.0) 


Black 


0 (0.0) 


3(1.2) 


56 (3.9) 


42 (3.7) 


Hispanic 


2(1.0) 


13 (3.5) 


52 (4.7) 


33 (5.2) 


Grade 8 


41 (1.4) 


27(1.5) 


19(1.0) 


13(1.0) 


Male 


41 (1.8) 


25 (2.5) 


18(1.9) 


16(1.3) 


Female 


41 (2.2) 


29(1.7) 


19(1.5) 


11 (1.4) 


White 


48(1.8) 


28(1.7) 


16(1.2) 


8 (0.8) 


Black 


18(3.6) 


25 (2.6) 


30 (3.5) 


28 (3.3) 


Hispanic 


22 (2.8) 


24 (3.5) 


25 (3.3) 


29 (4.5) 


Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Sample Geography Question 

Grade 12 — Atlas: Why Is Suez Canal Important ? 



Study the maps of the Middle East on pages 64 and 65 of the atlas. Using 
the maps, explain why the Suez Canal is both politically and economically 
important. 
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This extended constructed-response question required students to examine 
various maps of the Middle East and, using information in these maps, explain why the 
Suez Canal is both politically and economically important. The exercise required students 
to demonstrate an understanding of the importance of this waterway to the transportation 
of Middle East oil to international markets. 

For a “Complete” response, students were expected to indicate (or imply) the 
canal’s strategic location (the Suez Canal allows access to the Mediterranean and Red 
Seas) and give one significant political and one significant economic reason why this is 
so. Acceptable economic reasons specified that the canal allows shipping between Asia 
and Europe, without the extra cost of traveling around Africa; acceptable political reasons 
specified that access to the Suez Canal can be used as a bargaining tool by those who 
control it. 

An “Essential” response implied or indicated the canal’s strategic location and 
gave a significant political reason and significant economic reason with little or no 
explanation of the importance of location. A “Partial” response mentioned the canal’s 
strategic location but did not explain its significance economically or politically. 
Alternatively, it identified one significant reason with no reference to location. An 
“Inappropriate” response did not explain the strategic importance of the Suez Canal either 
politically or economically, nor did it cite location as significant. 

Actual student responses in each of these four rating categories are shown on the 
following two pages. 
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Sample “Complete” Response 



4. Study the maps of the Middle East on the pages 64 and 65 of the atlas. Using 
the maps, explain why the Suez Canal is both politically and economically 
important. 

cLAjL CoLrroJj AJZ. 

(Lad. jjpt± pen loodLa^oLXj- ^ 

'IjuzLbL Q<T , < 



p odLrXffO-tiA^ t r\ 




OLich 



<Ul 

Zu &qJqL ^oL ii Qq^ujJI 

CC^/\ dt S^LXX&h dJ?£^yZXldxJ&. V±<Ln£3< \ 

X&4- IaJOHlea. Q^-d rUsrcS- 



-dj 



-0±. 



1 ■ *" P 1 

Co- jdL 'Y^sO.djL a£ pOrZ-AL(hLa 



*a_ on, 



Jl£ &Lu^ 

'jfa. 

_aAc 



^j3cn •/ /VtOUvlAOj Z«v3^v> 

d Gx^L LU 



'T^U^c 



£Mr\ t Ac^A^JL QLiMI UACci. /m //q/ 

^-A^)oaj £j d od&iX 

oj -Ju. 



Qx^n L 



tin - h^o- 



±L&- dZLearjLA -pSKXMA ^ 

Oim\^VY^ £Jsp OpLsyrsP^l Ad /■ 

oXL 'LLs. Oj *acLt- O^o 4 

JUh^CU<^l\ * fll/LvvJj £/vtivr6\^Ofe z/^*Jks 

^juLtj^hd £ sfacbp- Gxxso/s Q— f* 

o n**L^ Cjolt^L. 
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Sample “Essential” Response 



4. Study the maps of the Middle East on pages 64 and 65 of the atlas. Using 
the maps, explain why the Sez Canal is both politically and economically 
important. v . , i 

■xa a ILrvK Cfl\vruMp ^ 



report; 



toko aA 0. u)ea 11J- i ^vdcW«0 

tomr^afUluj la ytKflAA- ihueA . 

1,1 Qf Aft CiLcivyO^ CxDC Aeft- 



u^oiAi 



.1/1 (y\ £r*\ Q^> 



Kj, M ty* S^tjQcy ytrdft 



Sample “Partial” Response 



1— 

4. Study the maps of the Middle East onpages 64 and (55 of the atlas. Using 
the maps, explain why the Suez Canalo is both politically and economically 
important. 

0 _ J\jL jL_2vl 0 Ul/xAwJI. ~lw( 1 A /rw\ 0 jL - 


\/ykTw 


-L-AWvA- . ) 


V 1 1 

0 fVuJc* 


* V*' 1 1 

o a dc 9 A/^jlXh (W\Cl 


Ars vlL#: o. ~ C&u-AjcAJU 


0 a. /A 4— 


(V^va /aV 1 . n/s L i7W\ JLxui r. 




ca \K/> l \) tA nr\ 9 1 


i 


t4- /T^/v^m \jl jdLos\ A M. 0 Q.CVC1 ( 


\ 


0^ Ao cvo vt- VvC — 

(l£pu>wv,K^O 

ytA^La 





Sample “Inappropriate” Response 



4. Study the maps of the Middle East on pages 64 and 65 of the atlas. Using 
the maps, explain why the Suez Canal is both politically and economically 
important. 

Ta'a V-TTAficfv'^.suiffOJcd dLt^ UJoia-. 
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To correctly answer this question, students had to examine a general political 
map of the Middle East showing the location of the canal, as well as a specialized map 
showing major oil fields and tracing the changing boundaries of the Sinai Peninsula, an 
area which borders the canal. Information about the amount of oil produced by OPEC and 
other leading petroleum producers was also shown on the same pages. Students had to 
synethesize all of this information and use it to explain why the Suez Canal is both 
politically and economically important. 

Information on student performance on this exercise is presented in Table 6.8. 
Almost 40 percent of twelfth-grade students provided a response that was scored as 
“Complete” or “Essential.” Thirty percent gave “Partial” responses, and another 30 
percent gave “Inappropriate” responses or omitted the exercise entirely. Most students 
grasped the importance of the location of the Suez Canal and were able to articulate this 
in some fashion. Many students were able to identify a reason for the economic 
importance of the Canal. 



Table 6.8 



Score Percentages for Atlas: Why Is 
Suez Canal Important? 



THE NATION S 
REPORT fp cipn 
CARD 








Complete 


Essential 


Partial 


Inappropriate 


Omit 


Grade 12 


8 (0.9) 


31 (1.4) 


30(1.2) 


18(1.3) 


12(1.0) 


Male 


11 (1.4) 


33 (2.1) 


29(1.9) 


17(1.8) 


11 0.1) 


Female 


6 (0.9) 


29 (2.2) 


32 (2.0) 


20(1.5) 


13(1.5) 


White 


10(1.2) 


34(1.7) 


32(1.5) 


16(1.7) 


8(1.0) 


Black 


1 (0.6) 


19(2.0) 


26 (3.1) 


29 (2.8) 


25 (2.8) 


Hispanic 


6(1.8) 


20 (2.6) 


26(3.1) 


23 (2.7) 


25(3.1) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Sample Geography Question 

Grade 12 — Sketch Graph to Show Continent Area 

The following exercise from the twelfth-grade assessment required students to interpret 
tabular data in the atlas and to use that information to construct a graph. 



Turn to the World Facts on the inside back cover of the atlas. Using the appropriate 
information from the tables, sketch bars on the blank graph below to show the 
approximate areas of the continents listed in square miles (to the nearest million). 

Provide a scale to explain your graph. 



Asia 






















Africa 






















Europe 






















Antarctica 























WORLD FACTS 



LONGEST RIVERS 



LARGEST LAKES 



CONTINENT 

NORTH AMERICA 

SOUTH AMERICA 

EUROPE 

ASIA 

AFRICA 

OCEANIA* 

ANTARCTICA 



sq. mi. 

9.355.000 

6.678.000 

4.056.000 

17.226.000 

11. 667 .000 

3.283.000 

5.500.000 



Miles 
. . . 4.132 


Kilometers 

6650 


LARGEST URBAN AREAS 


Population 


. , , 4.000 


6437 


Tokyo -Nbkohama. Japan 


. 25.434.CXX) 


. . . 3.915 


6300 


Mexico City. Mexico 


. 1 6,901. 000 


. . . 3.741 


6020 


Sao Paulo, Brazil 


. 14,911.000 


. . . 3.443 


5540 


NewMbrk.u.S 


. 14.598.000 


. . . 3.395 


5464 


Seoul. South Korea 


. 13.655,000 


. . . 3,381 


5409 


Osaka-Kobe* Kyoto. Japan 


. 13,562.000 


. . . 3.032 


4880 


Buenos Aires. Argentina 


. 10.750.000 


. . . 2.900 


4700 


Calcutta. India 


. 10,482.000 


, , , 2.761 


4444 


Bombay. India 


. 10,137.000 






Riode Janeiro. Brazil 


. 10.118.000 






Moscow. Russia 


. 9.873.000 


AREA 


Los Angeles. U.S 


. 9.638.000 


sq. mi. 


sq.km 


London. UK 


. 9.442.000 


. . . . 143.630 


372 000 


Paris. France 


. 6.633.000 


. ... 31.700 


82 103 


Cairo. Egypt 


. 6.595.000 


.... 26.626 


69 484 


Manila. Philippines 


6,485,000 


, . . . 23.050 


59 699 


Jakarta, Indonesia 


. 8,122.000 


. ... 22.300 


57 757 


Essen-Dortmund- Duisburg. Ger. ■ 


■ • 7.604.000 


. . . . 1 5,444 


40 000 


Tehran. Iran 


. 7.354.000 


. ... 12,700 


32 893 


Delhi. India 


. 6.993,000 


. ... 12.275 


31 792 


Shanghai. China 


. . . 6.698.000 


. ... 12.162 


31 499 


Chicago, U.S 


. . . 6.511.000 


.... 11.100 


26 749 










Percent 


Percent 


I AND AREA 


of world POPULATION 


of world 




sq.km 


land area ESTIMATE 


population 



24 235 800 
17816700 
10 507 800 
44 626 900 
30 225 400 
8 505 000 
14 248 700 



16.1 
1 1.9 
7.0 
29.7 
20.1 
5.7 
9.5 



416.000. 000 

268.000. 000 

685.000. 000 
3.009.000.000 

583.000. 000 

198.000. 000 



• 6.1 

5.2 

13.3 

56.3 

11.3 
3.6 



CONTINENT 


HIGHEST ELEVATION 
feet (meters) 


NORTH AMERICA 


Mt. McKinley. U S. 


SOUTH AMERICA 


20.320(6194) 

Ml. Aconcagua. Argentina 


EUROPE 


22.834 (6960) 

Mt. Elbrus. Russia 


ASIA 


16,510 (5642) 

Mt. Everest. Nepal-China 


AFRICA 


29.026(8848) 

Mt. Kilimanjaro. Tanzania 


OCEANIA* 


19.340 (5695) 

Jaya Peak. Indonesia 


ANTARCTICA 


16,500 (5030) 
Vinson Massif 




16.864 (5140) 



LOWEST ELEVATION 
feet (meters) below sea level 
Death Valley. U.S. 

262 (86) 

Peninsula Valdez. Argentina 
131 (40) 

Caspian Sea. Russia 
94(29) 

Dead Sea. Israel- Jordan 
1.312(400) 

Lake Assal. Djibouti 
512(156) 

Lake Eyre. Australia 
49 (15) 



‘Australia. New Zealand, and Pacific Islands 
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This short constructed-response question required students to review a number 
of tables presented on one page of the atlas, locate information showing the approximate 
areas of the continents in square miles, and translate this information into a bar graph. 
Students were also directed to provide a scale explaining their bar graph. 

For a “Complete” response, students were required to accurately draw to scale a 
bar graph representing the approximate areas of all four listed continents. A “Partial” 
response contained a graph that accurately displayed the relative proportions of the areas 
of the continents but was not correct in all details, or contained a graph that accurately 
reflected the sizes of two or three of the continents but with errors in relative proportion. 
An “Inappropriate” response did not provide a graph showing approximate areas of any of 
the continents listed, or provided a sketch but did not give a scale. Actual student 
responses in each of the three rating categories are included below. 

Sample “Complete” Response 



Asia 
Africa 
Europe 
Antarctica 

J. ft, to <2 \0 XL K up ft 20 IV 
32 . rru. ( to "tKO 'f\s~OJ\usf rr\i\\i'or»') 




Sample “Partial” Response 
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Sample “Inappropriate” Response 



Asia 


\ 


1 






0 


O 


o 


0 






Africa 


\ 


\ 


b 


> 
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Information on student performance on this exercise is presented in Table 6.9. A 
high percentage of twelfth-grade students attempted a response to this item; only 
1 percent did not respond at all. Seventy-three percent received at least a score of 
“Partial” for their response, and 26 percent provided a “Complete” response. The 
challenges in responding to this question included: 1) locating the table containing the 
relevant data, 2) appropriately selecting the column that referred to square miles rather 
than square kilometers, 3) rounding the land area values to millions of square miles, 4) 
drawing the bar length accordingly, and 5) providing an appropriate scale notation. The 
majority of students appeared able to locate the relevant information and to generally 
follow the appropriate conventions for a bar graph. However, substantially fewer were 
able to complete the full task successfully. 



Table 6.9 


Score Percentages for Sketch Graph to Show 
Continent Area 


. THE N 
'REPORT 
CARD 


lATlON’S 

raep 


Complete Partial Inappropriate Omit 



Grade 12 


26 (2.2) 


47(1.8) 


25 (1.7) 


1 (0.6) 


Male 


28 (3.1) 


46 (2.4) 


25 (2.8) 


1 (0.8) 


Female 


24 (2.7) 


49 (2.9) 


26 (2.2) 


2 (0.7) 


White 


28 (2.5) 


48(1.9) 


23 (2.0) 


1 (0.6) 


Black 


10(2.7) 


42 (4.7) 


44 (4.8) 


5 (2.7) 


Hispanic 


7(3.5) 


49 (8.1) 


44(7.1) 


0 (0.0) 



Standard errors of the estimated percentages appear in parentheses. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
1 994 Geography Assessment. 
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Summary 

The difficulty of the atlas questions is related not only to how well students are able to 
use an atlas to find information, but also to how well they can: extract information from 
the map or graphic stimulus, synthesize information from multiple maps or other sources 
of data, recognize patterns or trends, and analyze and provide reasons for their 
observations. 

The atlas sections proved difficult for students. In many ways, the results of the 
atlas sections paralleled those of the rest of the assessment. Students showed solid ability 
to answer questions that involved the use of a single, straightforward map or graphic. 
Students seemed to have some basic research skills. 

However, student performance at each grade level shows that many students still 
do not have the desired skills. There appears to be a need to provide students with more 
instruction on using an atlas, the central tool in geography. Students need exposure to and 
practice with extracting and synthesizing information from a variety of maps. In addition, 
students seem to need more instruction on the concept of scale, more training in analyzing 
trends and using outside information to explain the patterns or trends they are observing, 
and more practice in writing responses that clearly express their ideas. They also seem to 
have problems with maps other than straightforward or political depictions: relief and 
physical maps still present problems. 
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Conclusions 



The 1994 NAEP geography and U.S. history assessments incorporated a number of 
innovative features to expand the scope of the assessment beyond simple factual recall, to 
align the assessment tasks with instructional practices, and to make the assessment more 
engaging to students. This report has focused on one of these features — the use of a 
variety of tools and resources as stimuli for assessment questions. Many of the tools and 
resources selected for the assessment are particularly important in the fields of geography 
and history. For example, atlases and maps are inextricably tied to the study of 
geography, while primary source documents are of paramount importance in historical 
studies. Other materials, such as charts, graphs, and tables, are relevant to a wide range of 
disciplines. The use of these stimulus materials in the assessment allowed the assessment 
to more realistically measure the skills students should acquire through their study in the 
fields of geography and history. 

Students were asked to demonstrate a variety of skills in response to the 
questions that incorporated tools and resources as stimuli. For example, questions that 
asked about map conventions, or that required students to read information from a chart 
or graph, were among those that focused on direct comprehension of the stimuli. Some 
questions required factual recall of information not included in the stimulus, while other 
questions required interpretation and analysis, or application and evaluation of the 
stimulus information. In many questions, students were asked to use a combination of 
skills; for example, extracting information about a social trend from a chart and then 
bringing in outside factual or conceptual knowledge to explain the reasons for the trend. 
Student performance on the exercises described in this report should not, therefore, be 
interpreted solely as a reflection of students’ ability to read and understand the stimuli. 
Nevertheless, in general, the stimuli were essential features of the questions, and students 
had to read and understand them in order to respond correctly. 
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For the purposes of this report, questions that used a tool or resource as a 
stimulus were classified into one of six categories and discussed separately in the 
following groupings: 1 

Primary Source questions — which included questions that used text-based 
primary source documents as stimuli. 

Map questions — which included questions that used a map as a stimulus, 
exclusive of those that were part of the atlas block. 

Graphic questions — which included questions that used a chart, graph, 
table, or timeline as a stimulus, exclusive of those that were part of the atlas 
block. 

Atlas questions — which referred to a block of questions that used a Nystrom 
atlas as a resource. 

Photograph and Art questions — which included questions that used stimuli 
such as photographs, paintings, cartoons, and posters. 

Although the assessment questions have been categorized in terms of the stimuli they 
employed, it is important to recognize that the skills students were asked to apply crossed 
stimulus categories. For example, many questions in each category required that students 
recall factual or conceptual historical or geographical knowledge. Many questions across 
categories also required students to use analytical thinking skills to explain or interpret 
the historical or geographical phenomena depicted in the stimuli. 

Student Performance 

The complexity of the stimuli presented to students in the 1994 assessments reflected 
expert judgment regarding the level of expertise that well-prepared students ought to have 
developed with these tools and resources at each grade level. However, although student 
performance on the various questions showed that students at all grade levels had some 
familiarity with the different tools and resources, many students apparently found these 
questions quite difficult. 



1 Some of the questions in the 1994 geography and U.S. history assessments used stimuli that included 
materials from more than one tool and resource category. These questions, although very interesting, 
were few in number and have not been discussed in this report. O. O 
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Primary Source Questions 

Student performance on primary source documents varied depending on the factual 
knowledge that students were required to have in order to answer the questions, the 
format of primary source documents presented, and the level of analytical skills required 
to respond fully to the question. Based on the subset of questions that were presented to 
students across two grades, it was also evident that students in higher grades were better 
able to recognize and understand specific primary sources than those in lower grades. 

Although many students appeared to have had sufficient exposure to a variety of 
primary sources to allow them to identify the author of the document or interpret the 
information provided, they were less able to use the information in the documents to 
explain historical issues. Students were able to read and understand differing perspectives 
expressed in primary source documents such as letters, speeches, and news articles, but 
only at a superficial level. 

Map Questions 

Most students at all three grade levels seemed to have a basic understanding of maps and 
the information they provide. For example, the majority of fourth graders were able to 
locate the equator, the north pole, and the south pole; and almost half of eighth graders 
were able to identify parallels of latitude. However, it appeared that there were still many 
students with low levels of map literacy. For example, although many students were able 
to find the directions north, south, east, or west, they had greater difficulty with directions 
such as southeast or northeast. Similarly, many students understood the concept of scale 
but were less adept at using it. 

When it came to constructing their own maps, a substantial proportion of 
students had some knowledge about how to create and use a variety of map conventions. 
That is, they were able to draw a partially correct map from a textual description. The 
majority of students, however, were not able to draw, or were not careful about drawing, 
the completely correct rendition. 

Based on their performance, it appeared that many eighth- and twelfth-grade 
students had been prepared to interpret geographic features on maps and to understand 
how such features, whether natural or introduced, can impact people’s lives and the 
decisions they make. Many students, however, found it difficult to clearly explain their 
understanding of these concepts in writing. 
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Questions Based On Graphs, Charts, and Tables 

Many of the students at each of the grade levels were able to correctly extract data from 
simple graphs, charts, and tables. This suggests that many of these students were exposed 
to these tools and have a basic understanding of how to use them. However, the more 
complex the graphs, charts, and tables, the fewer the students who were able to 
understand the data presented in them. Furthermore, students’ performance suggested that 
it was difficult for most students to bring in outside knowledge to explain the data or 
phenomena illustrated in the graphic stimuli. 

Students in the eighth and twelfth grades were asked to draw a simple pie chart 
from tabular data. Most of the students were able to produce at least a partially correct pie 
chart. However, their performance indicated that their understanding of how to construct 
such a chart was not comprehensive. Many, for example, failed to label the pie segments 
correctly. 

Photograph and Art Questions 

Most students in all grade levels were able to extract direct clues to historical events or 
patterns from pictorial sources. Many were even able to connect the clues in photographs 
and art to historical contexts and make some simple inferences. At the twelfth grade, in 
particular, many students were able to understand artistic intent as well as the literal 
meaning of what they were seeing. As with questions utilizing other types of stimuli, 
however, when students were required to formulate explanations in their own words, they 
found the questions more difficult to answer correctly. 

Atlas Questions 

Based on their performance on the 1994 geography assessment, it might be inferred that 
students had some but perhaps not enough instruction in the use of an atlas as a resource. 
Although performance on the atlas questions was expectedly more advanced for students 
in higher grades, students at all grades appeared to need more practice extracting and 
synthesizing information from maps. In addition, the need for more instruction on the 
concept of scale and more training in analyzing trends and using outside information to 
explain observed patterns or trends was suggested by students’ performance on the atlas 
questions. Furthermore, student responses to the constructed-response atlas questions, and 
to the constructed-response questions in the assessment overall, suggest that students 
need more practice expressing their ideas in writing. 
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Summary 

Although the students who participated in the 1994 NAEP assessments in geography and 
U.S. history were familiar with the different types of tools and resources presented to 
them, for many students the familiarity lacked depth. That is, most students had some 
knowledge of what the tool or resource was and what it offered in terms of information, 
but few students were able to understand how to most effectively and appropriately 
interact with the tool or resource in order to provide a complete response. Also, the many 
questions requiring higher order thinking skills and the written expression of ideas were 
difficult, irrespective of the particular resource or tool involved. In addition, although 
factual knowledge questions were often easier than questions that asked for explanations, 
interpretations, or opinions, the level of difficulty still often depended on students’ 
exposure to the content knowledge required rather than on the particular stimulus used in 
the question. 

Tools and resources can invite students to engage more fully with the content of 
geography and history and can serve to increase students’ ability to think analytically. 
Furthermore, students are expected to be able to understand and use these tools and 
resources in everyday life as well as in their studies of geography and history. Therefore, 
it is imperative that students’ instruction include more exposure to and a more 
comprehensive treatment of these types of materials. 
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